Proper profiling of Continuity’s source code is useful for identifying code bottlenecks and functions that scale poorly. Python’s built in profiler, cProfile can effectively characterize the run time performance of Continuity. The pstats module analyzes the output from cProfile. This page describes two ways to use cProfile with Continuity. The preferred method is probably application dependent. cProfile can identify functions that are bottlenecks, but often the underlying problem is still difficult to identify. The python module line_profiler can give line-by-line timing metrics for specific functions.
To learn more about cProfile and pstats see the official Python documentation here
To learn more about line_profiler see the documentation and download site
cProfile in a Python Run Script
Perhaps the easiest way to use cProfile with Continuity is to incorporate it in a python script used to run Continuity in batch mode. In this manner, specific Continuity commands or groups of commands can be profiled.
The syntax for profiling from a script is as follows:
import cProfile
cProfile.run('function()', 'cont.profile')
The function() can be an individual continuity function or a function that calls a number of functions (see examples). Note that variables cannot be passed into an individual function. ‘cont.profile’ is the file that cProfile will save with the results. By default it is saved in the /pcty/ folder. It will need to be analyzed using the pstats module.
Example 1
In the below example, a single Continuity function is profiled.
import cProfile
import pstats
self.Load_File('/path/to/file/TestEP.cont6', log=0)
self.Send(None, log=0)
self.CalcMeshFast([('Calculate', None), ('Do not Calculate', None), ('Calculate', None), ('Angle change scale factors (for nodal derivs wrt angle change)', None)], log=0)
self.SinitElectrophys(log=0)
cProfile.run("self.SintElectrophys({'numGpuThreads':64,'plicitType':'Implicit','parallelLinearSolver':False,\
'solutions':{'writeFile': 0, 'counter': 20, 'tableResult': 0, 'saveSolutionFrequency': '100', 'renderResult': 0},\
'stateVarInputSelections':[],'stateVarDoTable':0,'parallelODESolver':False,'tstart':0.0,'useCuda':0,\
'stateVarOutputSelections':[],'serverKeyname':'electromech_exchange','stateVarList':'','fileName':'ProfileTest','useTrilinos':0,\
'aps':{'writeFile': 0, 'counter': 10, 'tableResult': 0, 'node_list': 'all', 'renderResult': 0},\
'stateVarListType':'collocation points','stateVarFrequency':1,'stateVarSelections':[],'dtout':0.1,'tlen':5,'reassemble_lhs':1,\
'ecgs':{'getHeartVector': False, 'writeFile': 0, 'counter': 10, 'tableResult': 0, 'renderResult': 0}}, log=0)", 'intEP.profile')
p = pstats.Stats('Cont.profile')
p.sort_stats('time').print_stats(50)
Example 2
import cProfile
import pstats
def run_continuity(self):
#define parameters
fName = 'ProfileTest'
tstart = 0.0
duration = 5
stepsize = 0.1
self.Load_File('/path/to/file/TestEP.cont6', log=0) self.Send(None, log=0)
self.CalcMeshFast([('Calculate', None), ('Do not Calculate', None), ('Calculate', None), ('Angle change scale factors (for nodal derivs wrt angle change)', None)], log=0)
self.SinitElectrophys(log=0)
self.SintElectrophys({'numGpuThreads':64,'plicitType':'Implicit','parallelLinearSolver':False,\
'solutions':{'writeFile': 0, 'counter': 20, 'tableResult': 0, 'saveSolutionFrequency': '100', 'renderResult': 0},\
'stateVarInputSelections':[],'stateVarDoTable':0,'parallelODESolver':False,'tstart':tstart,'useCuda':0,\
'stateVarOutputSelections':[],'serverKeyname':'electromech_exchange','stateVarList':'','fileName':fName,'useTrilinos':0,\
'aps':{'writeFile': 0, 'counter': 10, 'tableResult': 0, 'node_list': 'all', 'renderResult': 0},\
'stateVarListType':'collocation points','stateVarFrequency':1,'stateVarSelections':[],'dtout':stepsize,'tlen':duration,'reassemble_lhs':1,\
'ecgs':{'getHeartVector': False, 'writeFile': 0, 'counter': 10, 'tableResult': 0, 'renderResult': 0}}, log=0)
cProfile.runctx('run_continuity(self)',globals(),locals(),'Cont.profile')
p = pstats.Stats('Cont.profile')
p.sort_stats('time').print_stats(50)
Invoke cProfile when Opening Python
cProfile can monitor an entire Continuity session instead of a specific function(s). The ./continuity file must be edited when python is invoked to include cProfile (around line 212). The general syntax is as follows:
python -m cProfile -o Cont.profile RunContinuity.py
Where ‘Cont.profile’ is the name of the output file. The specific change to the ./continuity file would therefore be:
exec $python $pyflags -m cProfile -o Cont.profile $continuity $@
Other cProfile Tools
Gprof2Dot provides a neat graphical visualization to the cProfile output.
line_profiler in Continuity
This should be updated as I figure it out more!
-
Download line_profiler to pcty/MGLtools/lib/python2.5/site-packages/ from the download site
wget https://pypi.python.org/packages/source/l/line_profiler/line_profiler-1.0b3.tar.gz --no-check-certificate
- Untar/extract the tar ball
- source ./mglinit in the cont_dev/ directory
- python setup.py install
import line_profiler
prof = line_profiler.LineProfile()
prof.add_function(mesh.CalcTensorAtGaussPoint)
prof.runctx('self.build_matricies()',globals(),locals())
prof.print_stats()
prof.dump_stats('filenameout.lprof')
To convert the output file to a human-readable text file use the following syntax from the command line:
python -m line_profiler filenameout.lprof > filenameout.txt