Image Landmarks

Description

• This tutorial will explain how to use the Landmarks feature within the Image Editor portion of Continuity.
• The general purpose of using landmarks is not to just be able to identify and save the location of places of interest on certain images, but to use landmarks to be able to rotate and translate (i.e., change origin location) a stack of images as desired.

• The images and necessary data can be obtained from this file: healthyRabbitLandmarkTutorial.zip.

Start Continuity

• Launch the Continuity Client

• On the About Continuity startup screen

  • check the fitting module box under Use Modules:

Open Image Editor and Identify Landmarks

• Using the menu headers at the top of the Continuity screen select: Fitting→Edit→Images…

• This will open the image editor in a new window, from which you can then load in the stack of images you would like to work with: File→Open Stack. Browse to the healthyRabbitLandmarkTutorial/imageStack directory that contains the stack of images and hit ‘OK’. Note, you likely will not see any files actually listed in that directory, that is normal. Currently, Continuity is expecting the stack of images to be 256×256 pixels in size, and sequentially arranged such that when imported in that order they will form the proper 3D representation of the heart, or whatever anatomy the images contain. Continuity is not currently able to figure out the exact arrangement of the images are on it’s own, and so it expects the user to have already put them into the proper arrangement.

• Once the images are loaded, if it has not been done for you, you should specify the scaling of the image set, i.e., what length does 1 pixel in the image relate to in the real world in the x and y directions, and the z direction is how far apart each individual image is. Input these values on the Transformation tab under scale and hit return after entering those numbers for the numbers to take effect. Once you know the translation, scale and rotation, you can save that information into an xml file, along with the names of all the images, and then on subsequent stack imports, the translation, scale and rotation will be set automatically. That is the case with the attached data.

• For our example, under scale, enter x = 0.14, y = 0.14, z = 1, to properly scale the pixel coordinates by the voxel sizes (in millimeters).

• Now you can start identifying the landmarks you are interested in. By default, there are 17 landmarks already listed for you to identify; however, you are free to load in a file containing the names of your own landmarks should you want. The file format is simply:

LandmarkName, xCoord, yCoord, zCoord

• where xCoord, yCoord and zCoord are in normalized image based coordinates (e.g., xCoord value would be pixel number in x direction divided by the total number of pixels in the x direction), but you can set each of them to ‘undefined’ on the initial import. An alternative approach is to just save the existing set of landmarks to a file, and modify that file to suite your needs.

  NOTE: For the default set of landmarks, there are 5 that are used in the transformation from the initial orientation to the desired orientation. You can designate which landmarks are critical for that by placing an ‘*’ at the end of the landmark’s name. However, to actually get the origin and rotation computations to use the exact landmarks needed, currently the actual python code would need to be modified. In the future the user will be able to have more control over which landmarks are used for the transformation process. Currently landmarks in positions 1, 3 and 16 are used for the computation of the new origin, and landmarks 5, 13 and 16 used to compute the axis rotation. See below for more details on how the origin and rotation are computed.

• To identify a landmark, first click on the Point button on the far right side of the image window, then navigate to the image that contains the landmark you want to identify, using the Image Slider (also on the far right). After making sure you have selected the proper landmark name in the list of landmarks, click on the area of the image that contains your landmark. You should then see a yellow dot appear on the image and the coordinate values appear in the coordinate window to the right of the list of landmarks. There are two sets of coordinate values: world and image. The image coordinates show the x,y,z location of your landmark, where x and y are the pixels locations (ranging from 1 to 256), and z corresponds to the image number in the stack of images. The world coordinates are the image coordinates after being adjusted by any information contained on the ‘Transformation’ tab (e.g., scale factor). Continue this procedure until you have at least identified the required landmarks.

• You can either try your luck at selecting the landmarks your self, or you can load the included landmark file, healthyRabbitLandmarkTutorial/landmarks.txt which has all but 5 landmarks already defined, the remainder left for the user to define.

  • 

Transform Image Stack Based on Selected Landmarks

• Once you have identified at least the required landmarks, a Transform Landmarks button will appear underneath the coordinate display window. When clicking on this button, Continuity will:

  • Compute a new origin for the stack of images
  • Rotate the images to the required orientation

• The final result after defining the new origin and rotating the image stack is that the new x-axis will go halfway between the left ventricle (LV) midseptal basal endocardium and the LV lateral basal endocardium, and directly through the LV apical endocadium, with positive x-direction going from the base region through the apex. The positive y-axis goes through the right ventricle and the z-axis is perpendicular to both x and y. You can turn on the axis lines by: View->Toggle Axis

  • 

• Currently, the origin is simply computed by finding the mid-point between the LV midseptal basal endocardium and the LV lateral basal endocardium and then 1/3 of the way down towards LV apical endocadium. The formula being:

  • 

  Where z(1) = LV_midseptal_basal_endocardium vector, z(3) = LV_lateral_basal_endocardium vector and z(16) = LV_apical_endocadium vector.

• Rotation is handled by the given formula (from http://en.wikipedia.org/wiki/Rotation matrix#Rotation matrix given an axis and an angle):

• Since we already know what the column vectors are, we want to solve for the axis and angle. The column vectors are defined by:
  • e1 is a vector that points from origin to apex
  • e2 is a vector that points from the origin to midseptum
  • e3 is a vector that is perpendicular to both e1 and e2
  • Their formulas being:

    

  • Where z(5) is RV_midseptal_basal_endocardium vector, z(16) is the LV_apical_endocardium vector, and z(13) is the RV_midseptal_apical_endocardium vector.
• It then follows that the axis and angle can be obtained via:

  • 

  We do it this way as the rotation form on the Transformation tab takes the axis and angle values directly as entries, and we can then feed these values directly into the form and the rotation will then be handled via that rotation matrix.

• After performing the transformation and all of the landmarks have been identified, you can create a mesh from the resulting data by selecting Landmarks under Calculate and Send to Mesh under Results on the Calculate tab. This will:

  • Convert the landmarks from cartesian coordinates to prolate spheroid coordinates
  • Interpolate values from the initial 17 landmarks to form the full set of 50 landmarks.
  • Put the resulting lambda, mu and theta values into the node form (i.e., Mesh→Edit→Nodes) for all 50 nodes .
  • Compute the focus of the prolate spheroid and put that into the Mesh→Coordinates form and define the new coordinate system to be prolate spheroid.
  • Define the node and element connections in the Mesh→Edit→Elements form.

Render the Mesh

• This process does not choose a basis function for you, so you must pick one before trying to do any rendering. You define a basis function from Mesh→Edit→Basis. The following works for the default landmark listing: Choose Langrange Basis Function→3D→Linear-Linear-Linear and then click Add and you should then see both Linear-Linear-Linear Lagrange 3*3*3 and Linear-Linear Lagrange 3*3 basis functions listed.

• After choosing a basis function, you must now associate that basis function with your nodes, if not already done for you. From Mesh→Edit→Nodes, see if any basis function is defined for each of the 3 coordinates, and if not, select Linear-Linear-Linear Langrange for each of the 3 coordinates.

  • 

• To see the final result, you just need to render the nodes/elements that we have computed. You most likely just want to see the resulting surfaces, so do Mesh→Render→Elements and then select the surfaces button and finally hit Render. If you want to see where the landmarks are, you can render the nodes via: Mesh→Render→Nodes, and this will display all 50 nodes. Use View→Show Open Mesh if you want to make any adjustments to size/colors of things that have been rendered.