Understanding the Continuity Source

Intro

Languages

• Continuity is implemented primarily in Python, Fortran, and C. For an experienced python programmer, writing and maintaining python code is probably 100 times easier than a compiled language. However, python code probably runs about 100 times slower than a compiled language. Thus in Continuity we make use of very high level python for parts of the application which are not highly performance dependent and lower level compiled languages otherwise.

  • Python. Python is an interpreted language, often referred to as a “scripting” language. Python is dynamic, very high level, easy to learn and use and the majority of Continuity is implemented in this language. Only the most computationally intense parts of the application are written in lower level compiled language such as Fortran or C. At the time of writing, we use python2.5. There are newer version of python (notably python2.6 and python3) but we cannot upgrade until our main dependency (The MGLTools library) also upgrades.

  • Fortran. Our oldest “legacy” code is written in Fortran which is responsible for most of the number crunching. Compiled Fortran code runs at least 100x faster than its python equivalent, which is important to keep in mind when making design decisions about engineering the software. Most of our Fortran is Fortran77 compatible, although there may be some parts that use dynamic memory allocation and require a Fortran95 compiler. Generally, though, we allocate dynamic memory in python and pass the data structures for use in Fortran. We use the Intel Fortran compiler and we’ve gone from version 8 to version 11 over the years without too much difficulty. The g95 compiler also has been used to compile Continuity.
  • C/C++. Some of our newer or dynamically generated parts of Continuity have been implemented in C. We generally see similar performance to the compiled Fortran code.

Tools and Libraries

• MGLTools. Continuity makes extensive use of some libraries distributed as part of MGLTools, a sister project also funded by NBCR. In particular, we use:

  • DejaVu – a python based rendering engine built on top of OpenGL

  • ViewerFramework – a framework that we use to manage the menus, automatic script generation, etc.

• f2py – a tool used to generate C bindings to enable communication between Fortran and python
• swig – a tool to generate C bindings to ease communication between python and C or C++.

• Tkinter – a library for binding GUIs (Graphical-User-Interfaces) in python. These GUIs are platform independent and run on Windows, Linux, and the Mac, but don’t look particularly native on any platform.

• Pmw – (Python Mega Widgets) is a python library built in top of Tkinter to ease the creation of GUIs.

• numpy – An excellent numerical library for python.

• Numeric – A deprecated library that served the same purpose as numpy. We attempted to remove all references to this library when we switched to numpy although some references may still remain.

Client-Mediator-Server Architecture

Continuity was designed as a Client-Mediator-Server architecture. This means that there is a front-end user interface to input and view data and a back-end server which acts on or manipulates this data. In the early days of Continuity the server could only run on the IRIX operating system while the client ran on Windows. However, now both the client and server can run on Windows, Linux, or the Mac and can run in a single process to avoid communication overhead. The client-server distinction, though, still remains throughout Continuity as is evident even at the user interface level where a user must send data before operating on it even when when running the client and server in a single process.

The client and the server can communicate using python sockets when run as separate processes. See <continuity>\pcty\cont_classes\ContCom.py to see how this works.

The mediator is another layer responsible for taking high level python data structures (such as dictionaries) and transforming them into a representation which is compatible with Fortran or C (such as data arrays). The mediator lives in various files in the pcty/cont_server directory such as Nodes.py, Elements.py, Mesh.py, etc. Other components of the mediator exist in pcty/server/problem directory and its subdirectories.

Source Structure

• pcty – The pcty directory (which stands for Python ConTinuitY) contains all the python scripts which make up Continuity.

  • cont_classes – cont_classes contains most of the mediator used for transforming python data structures into a format compatible with Fortran. It also contains classes to hold the data created by the various forms or user interfaces. For example, the NodesData.py script contains the NodesData() class which stores data generated by the NodesForm() class found in NodesForm.py

  • server
    • problem – contains scripts relevant for both biomechanics and electrophysiology problems. The Problem() class (in problem.py) is the parent class from which both Biomechanics() and Electrophysiology() classes are derived.
      • biomechanics
      • electrophysiology
  • client

    • OpenMesh

    • forms – All the various forms (such as NodeForm.py) can be found in this directory. Basically, it contains the majority of the GUI.

    • *command.py – All the files ending in command.py define the menu system used by Continuity.
• src – contains the Fortran and C source code.
  • fsrc – Miscellaneous Fortran source that is used by several different fortran modules.
  • csrc – Various C based submodules
  • mesh – Fortran mesh subroutines
  • nodes – Fortran nodes subroutines
  • data – Fortran fitting subroutines
  • fitting – deprecated
  • ep – Fortran electrophysiology f2py interface files
  • electrophys – Fortran electrophysiology subroutines
  • bm – Fortran biomechanics f2py interface files
  • biomechanics – Fortran biomechanics subroutines
  • include – a variety of header files used to define “common blocks”, basically Fortran global variables.

Continuity Variables

For a list of the variables used in Continuity that is (mostly) up to date see, this page.

Where’s main()?

Python does not strictly require a main() function like C or C++. However, ContinuityClient.py is the main script which launches Continuity and the Continuity() class inside of gui_client.py is the main instance of the Continuity Client. Also ContinuityDaemon.py is the main script which can be used to launch the Continuity Server, and Cont_Server() in cont_server.py is the main class for running the a Continuity server.

Client

Open Mesh and DejaVu

Continuity has two complementary rendering engines both built on top of OpenGL. OpenMesh can be found in pcty/client/OpenMesh and consists of pairs of files which implement the user interface and rendering for drawable objects. For example, Lines.py contains the class responsible for drawing lines and LinesGUI.py contains the class responsible for managing the GUI presented to the user for manipulating rendered lines. The OpenMesh() class (in OpenMesh.py) is responsible for managing the various rendered objects.

The Shell

• A key component part of Continuity is the python shell accessible via the top left button in the toolbar under the File menu. From this shell you can type any python command or script. However, the most powerful use of this shell is the ability to dynamically script Continuity at run time. This allows a poweruser to check or change the state of data and better debug a problem. As always, python allows you to inspect data attributes with the dir() command.

Here are some important data structures you can reference from the Continuity shell:

• self – The root of continuity can be accessed via the “self” variable which is a reference to the instance of the Continuity class (as defined in gui_client.py) which is a child class of ViewerFramework an MGLTools class.

• self.stored_data. Contains all the persistent data from Continuity forms or dialogs. For example, self.stored_data.nodes.obj contains the data from the nodes form. self.stored_data is actually what gets serialized or pickled when the user saves a model to disk.

• self.om. A reference of the OpenMesh class

• self.om.rolist. A list of currently rendered objects.

The Command Log

As a Continuity user opens forms, executes commands, etc. there is a log of commands automatically managed by ViewerFramework (the library that Continuity is built upon). You can see this log by clicking on the “Message Box” icon which is the second icon in the toolbar. You can save this log to a file by selecting File > Scripts > Create Script…

Continuity allows the user to run a script (either generated by automatic log, or created by hand) by selecting File > Scripts > Run Script…

Server

The Make system

• autoconf – reads configure.in and builds configure
• ./configure – reads the various platform independent Makefile.in files and builds platform dependent Makefiles
• dynamicMake.py – builds dynamically generated modules.

Windows challenges

• Because our make system needs a POSIX environment (i.e. the unix shell) Windows requires the installation of program that allows you to run Unix commands in order to compile. We use mingw for this purpose.

Dynamically Generated Modules

Continuity allows the user to specify certain aspects of the model in a high level pythonesque syntax which it can then use to dynamically author and compile a lower level C representation. We currently use this technique with the following modules:

• ionic model editor (electrophysiology)
• constitutive model editor (biomechanics)
• material coordinates editor (mesh)

Other Topics

How can python communicate with C or Fortran codes?

There are several different ways to use python together with a compiled language. We will focus on the ways we do this in Continuity, although we do not claim that this is the only way (see ctypes for an alternative).

Python is capable of importing a C shared library that conforms to a specific interface. A novice programmer is accustomed to the idea of compiling source code into an stand-alone executable. However, compilers can also build libraries rather than executables that can be used by programmer. For example, passing the –shared flag to the gcc compiler will tell the compiler to build a shared library than an executable.

If this shared library specifies the proper interface then python will be able to call the functions defined in the library. There are several ways to define the proper interface

1. Define the interface manually. Although this may sound difficult, with a little practice it’s not that difficult, especially for a small library. See <continuity>\src\csrc\region_detect\region_detect.cpp for an example of a simple C file which defines an interface that python will be able work with. See its accompanying makefile to see how this is compiled.

2. Automatically generate the interface using swig. When dealing with sufficiently complex C or C++ code, it is often easiest to allow an automatic interface generator to do all the work for you. See <continuity>\src\csrc\specialsort for an example of a module which makes use of swig for its interface.

3. f2py can also build interfaces. We use f2py to build all the interfaces for our Fortran modules. By convention, we distinguish the files with f2py directives from normal Fortran subroutines by prepending file names with “p_”. For example p_upnodes.f contains f2py directives to wrap the subroutine upnode() defined in upnodes.f.

How can I get started?

With over 100K lines of source code, understanding the intricacies of Continuity will take some time and experience. I would recommend that you start by following the logic and data flow of a certain user action. For example, if the user selects Mesh > Render > Elements… something like this happens:

• meshCommands.py – RenderElems:guiCallback() is called which creates the renelemForm().

• The results of the renelemForm form are passed to self.doitWrapper() (in the Command.py parent class) which then calls doit()
• doit() sends data to the server with this line: self.vf.send_cmd( [‘ren.Elements’, args] )
• cont_classes/Render.py – the Elements() function is called on the mediator (which resides in the server process).

• p_reelems() is called which is a Fortran server function defined in src/render/p_reelems(). This is an f2py interface file which wraps reelems.f found in src\fsrc\reelems.f

• Data is returned from reelems() to p_relems() to Elements() to doit() in RenderElem() of meshCommands.py.

• Depending upon the type of data, parseQuads(), parseTriangles(), or parseLines() is called which returns an OpenMesh objects

• Finally, self.vf.om.addRenderObject() is called (a function of the OpenMesh() class) to add this new OpenMesh object to the list of rendered objects.

As an exercise, try to create this sort of program flow for various menu commands.

How do I add something to a menu?

First, locate, the appropriate *Commands.py file. So if you want to add something to the Mesh menu, open MeshCommands.py located in pcty/client/. Follow the example of other commands to set this up properly. The main thing to keep in mind is that there is a difference between the guiCallback() and the doit() functions. guiCallback() is what actually gets called when the user selects your new menu. At the end of guiCallback() you should call self.doitWrapper() and pass it the data that you want to show up in the command log. doitWrapper will automatically call doit(), after it has written the appropriate information to the log file. If you need more information about this, hopefully the guys who develop the MGL Tools can help you out.

How do I save data for a new form (Dialog box)

• Add an entry in client/StoredData.py’s store() function

Unit Tests

We have over 120 unit test cases which can be run from within Continuity (Help > Run Unit Tests…). See pcty/tests for more details.