Detailed Description

This module provides RMF I/O.

The library provides support for the RMF file format for storing hierarchical molecular data (such as atomic or coarse grained representations of proteins), along with markup, including geometry and score data.

The library uses the HDF5 library to manage the data on disk. Other backends (eg mmCIF) could be used, if desired.

See RMF file format for more information about the files and RMF data categories for standard data.

The RMF library

The RMF library provides an intermediate level interface to facilitate I/O of RMF data into and out of programs. The primary classes of interest are RMF::RootHandle representing the root of an RMF hierarchy and RMF::NodeHandle representing a node in the hierarchy.

The file is automatically closed when the last handle to it is destroyed.

0 Basics

The library defines many classes, some of which are implemented using C++ templates. Every class support output to a std::ostream in C++ and conversion to str in Python. In addition, every class can be compared to other instances of the same class and can be inserted in hash tables both in C++ and Python. The methods necessary to support these things are omitted for brevity.

Template classes are, in general parameterized by one of two things

Arity, meaning whether they relate to a single node, a pair of nodes etc
type, meaning whether they relate to integers, floats etc For any class that is parameterized by Arity, typedefs (and Python types) are provided for 1 through 4 nodes, with names like Category, PairCategory, TripletCategory, QuadCategory. For classes parameterized by type, typedefs and Python classes are provides for all the standard types with names like IntKey, FloatKey etc. Some classes, like Key are parameterized by both, so the full cross product of types is provided with names like PairFloatKey.

In addition, there is a typedef for each type for managing lists of the objects. For example, a list of Category objects is passed using a Categories type. It looks like a std::vector in C++ and is a list in Python.

1 Associations

The RMF wrapper has the concept of an association between nodes in its hierarchy and objects in the program accessing. The methods RMF::RootHandle::get_node_handle_from_association(), RMF::NodeHandle::set_association() and RMF::NodeHandle::get_assocation() can be used to take advantage of this. The idea is that one can store pointers to the programatic data structures corresponding to the nodes and so avoid maintaining ones own lookup table.

2 Accelerated I/O

Traversing large RMF files can be slow, especially when doing so in scripting languages. The library provides a few methods RMF::get_values() to accelerate this process when loading values for each of a number of frames. Other such batch methods can be added as appropriate.

3 Parallel I/O

If RMF::RootHandle::flush() has been called since the last change, it is safe to read the file from another process. Writing from more than one process is not supported. Nor is reading or writing from more than one thread of the same program.

4 Invariants

Currently, there is little explicit checking of invariants between attributes in the RMF file. An extensible framework for checking invariants on file close and open will be added.

5 Other language bindings

The RMF library currently supports C++ and Python. The API is written so that SWIG can be used to easily generate bindings for most languages. The two main exceptions are C and Fortran. Until the SWIG C target support is finished, these can be supported by writing a simple C API manually, probably a weeks work.

6 HDF5 wrappers

The library provides a simple wrapper for the HDF5 library through the HDF5Group, HDF5DataSetD, HDF5File and helper classes.

Helper programs

rmf_show prints out the hierarchy written to the file.

rmf_pdb converts an rmf file to or from a PDB file, assuming all hierarchies in the rmf file are atomic resolution.

rmf_xml converts an rmf file to an XML files that can be opened in an XML viewer (eg Google Chrome or Firefox). These viewers support collapsing of subtrees, which makes it much easier to get around large hierarchies.

See the RMF tools application for IMP-related helper programs.

Examples:

Author(s): Daniel Russel

Version: SVN.r14091

License: LGPL. This library is free software; you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

Publications:

Daniel Russel, Keren Lasker, Ben Webb, Dina Schneidman, Javier Velázquez-Muriel, Andrej Sali, “Putting the pieces together: integrative structure determination of macromolecular assemblies”, PLoS Biology, 2012. This paper provides an overview of the key concepts in IMP and how to apply them to biological problems.
Frank Alber, Friedrich Foerster, Dmitry Korkin, Maya Topf, Andrej Sali, “Integrating diverse data for structure determination of macromolecular assemblies”, Annual Review of Biochemistry, 2008. This paper provides a review of the integrative structure determination methodology and various data sources that can be used.

Classes
class	Atom
class	AtomConst
class	AtomConstFactory
class	AtomFactory
class	Ball
class	BallConst
class	BallConstFactory
class	BallFactory
class	CategoryD
class	Chain
class	ChainConst
class	ChainConstFactory
class	ChainFactory
class	Colored
class	ColoredConst
class	ColoredConstFactory
class	ColoredFactory
class	Copy
class	CopyConst
class	CopyConstFactory
class	CopyFactory
class	Cylinder
class	CylinderConst
class	CylinderConstFactory
class	CylinderFactory
class	Diffuser
class	DiffuserConst
class	DiffuserConstFactory
class	DiffuserFactory
class	Domain
class	DomainConst
class	DomainConstFactory
class	DomainFactory
class	Exception
class	FileConstHandle
	A handle for a read-only RMF file. More...
class	FileHandle
	A handle for an RMF file. More...
class	HDF5ConstAttributes
class	HDF5ConstDataSetD
class	HDF5ConstFile
class	HDF5ConstGroup
class	HDF5DataSetD
class	HDF5DataSetIndexD
class	HDF5File
class	HDF5Group
class	HDF5Handle
	Make sure an HDF5 handle is released. More...
class	HDF5MutableAttributes
class	HDF5Object
class	HDF5SharedHandle
	Share an HDF5 handle. More...
class	IntermediateParticle
class	IntermediateParticleConst
class	IntermediateParticleConstFactory
class	IntermediateParticleFactory
class	InternalException
class	IOException
class	JournalArticle
class	JournalArticleConst
class	JournalArticleConstFactory
class	JournalArticleFactory
class	Key
class	NodeConstHandle
	A handle for a particular node in a read-only hierarchy. More...
class	NodeHandle
	A handle for a particular node in the hierarchy. More...
class	NodeIDD
class	NodeSetConstHandle
	A handle for a set of particles. More...
class	NodeSetHandle
	A handle for a set of particles. More...
class	Particle
class	ParticleConst
class	ParticleConstFactory
class	ParticleFactory
class	Residue
class	ResidueConst
class	ResidueConstFactory
class	ResidueFactory
class	RigidParticle
class	RigidParticleConst
class	RigidParticleConstFactory
class	RigidParticleFactory
class	Score
class	ScoreConst
class	ScoreConstFactory
class	ScoreFactory
class	Segment
class	SegmentConst
class	SegmentConstFactory
class	SegmentFactory
class	StaticAlias
class	StaticAliasConst
class	StaticAliasConstFactory
class	StaticAliasFactory
class	Typed
class	TypedConst
class	TypedConstFactory
class	TypedFactory
class	UsageException
class	Validator
class	vector

Standard module methods
All `IMP` modules have a set of standard methods to help get information about the module and about file associated with the modules.
std::string	get_module_version ()
std::string	get_module_name ()

Batch data access

These methods provide batch access to attribute data to try to reduce the overhead of repeated function calls.

The missing_value argument is a placeholder that can fill in for values which are not found in the respective node.

Note:: These methods are experimental and subject to change.

RMFEXPORT Floats get_values (const NodeConstHandles &nodes, FloatKey k, unsigned int frame, Float missing_value=std::numeric_limits< double >::max())

Helper functions
These functions help make working with keys and categories easier, by allowing you to assume that the key/category is always there.
template<int D>
CategoryD< D >	get_category_always (FileHandle fh, std::string name)
CategoryD< 1 >	get_singleton_category_always (FileHandle rh, std::string name)
CategoryD< 2 >	get_pair_category_always (FileHandle rh, std::string name)
CategoryD< 3 >	get_triplet_category_always (FileHandle rh, std::string name)
CategoryD< 4 >	get_quad_category_always (FileHandle rh, std::string name)
template<class TypeT , int D>
Key< TypeT, D >	get_key_always (FileHandle fh, CategoryD< D > cat, std::string name, bool per_frame=false)
Key< TypeTraits, 1 >	get_type_key_always (FileHandle rh, CategoryD< 1 > cat, std::string name, bool per_frame=false)
Key< TypeTraits, 2 >	get_type_key_always (FileHandle rh, CategoryD< 2 > cat, std::string name, bool per_frame=false)
Key< TypeTraits, 3 >	get_type_key_always (FileHandle rh, CategoryD< 3 > cat, std::string name, bool per_frame=false)
Key< TypeTraits, 4 >	get_type_key_always (FileHandle rh, CategoryD< 4 > cat, std::string name, bool per_frame=false)

Data set names
The RMF format stores various pieces of data in data sets and attributes attached to the HDF5 group which is acting as the root. These functions return the names for the various data sets.
String	get_node_data_data_set_name ()
	Get the name of the data set storing the data about each node.
String	get_node_name_data_set_name ()
	Get the name of the data set storing the name for each node.
String	get_bond_data_data_set_name ()
	Get the name of the data set for storing bonds.
String	get_set_data_data_set_name (int arity)
	Get the name of the data set for storing bonds.
String	get_category_name_data_set_name (int arity)
	Get the name of the data set for storing category names.
String	get_key_list_data_set_name (std::string category_name, int Arity, String type_name, bool per_frame)
	Get the name of the attribute which lists all the keys of the category.
String	get_data_data_set_name (std::string category_name, int arity, String type_name, bool per_frame)
	Get the name of the data set for storing a particular type of data.

Typedefs
typedef vector< AtomConstFactory >	AtomConstFactories
typedef vector< AtomConst >	AtomConsts
typedef vector< AtomFactory >	AtomFactories
typedef vector< Atom >	Atoms
typedef vector< BallConstFactory >	BallConstFactories
typedef vector< BallConst >	BallConsts
typedef vector< BallFactory >	BallFactories
typedef vector< Ball >	Balls
typedef vector< ChainConstFactory >	ChainConstFactories
typedef vector< ChainConst >	ChainConsts
typedef vector< ChainFactory >	ChainFactories
typedef vector< Chain >	Chains
typedef vector < ColoredConstFactory >	ColoredConstFactories
typedef vector< ColoredConst >	ColoredConsts
typedef vector< ColoredFactory >	ColoredFactories
typedef vector< Colored >	Coloreds
typedef vector< CopyConstFactory >	CopyConstFactories
typedef vector< CopyConst >	CopyConsts
typedef vector< CopyFactory >	CopyFactories
typedef vector< Copy >	Copys
typedef vector < CylinderConstFactory >	CylinderConstFactories
typedef vector< CylinderConst >	CylinderConsts
typedef vector< CylinderFactory >	CylinderFactories
typedef vector< Cylinder >	Cylinders
typedef vector < DiffuserConstFactory >	DiffuserConstFactories
typedef vector< DiffuserConst >	DiffuserConsts
typedef vector< DiffuserFactory >	DiffuserFactories
typedef vector< Diffuser >	Diffusers
typedef vector < DomainConstFactory >	DomainConstFactories
typedef vector< DomainConst >	DomainConsts
typedef vector< DomainFactory >	DomainFactories
typedef vector< Domain >	Domains
typedef vector< FileConstHandle >	FileConstHandles
typedef vector< FileHandle >	FileHandles
typedef herr_t(*	HDF5CloseFunction )(hid_t)
	The signature for the HDF5 close functions.
typedef HDF5ConstAttributes < HDF5Object >	HDF5ConstDataSetAttributes
typedef vector< HDF5File >	HDF5ConstFiles
typedef HDF5ConstAttributes < HDF5Object >	HDF5ConstGroupAttributes
typedef vector< HDF5Group >	HDF5ConstGroups
typedef vector< HDF5File >	HDF5Files
typedef HDF5MutableAttributes < HDF5ConstGroup >	HDF5GroupAttributes
typedef vector< HDF5Group >	HDF5Groups
typedef vector < IntermediateParticleConstFactory >	IntermediateParticleConstFactories
typedef vector < IntermediateParticleConst >	IntermediateParticleConsts
typedef vector < IntermediateParticleFactory >	IntermediateParticleFactories
typedef vector < IntermediateParticle >	IntermediateParticles
typedef vector < JournalArticleConstFactory >	JournalArticleConstFactories
typedef vector < JournalArticleConst >	JournalArticleConsts
typedef vector < JournalArticleFactory >	JournalArticleFactories
typedef vector< JournalArticle >	JournalArticles
typedef vector< NodeConstHandle >	NodeConstHandles
typedef vector< NodeHandle >	NodeHandles
typedef vector < ParticleConstFactory >	ParticleConstFactories
typedef vector< ParticleConst >	ParticleConsts
typedef vector< ParticleFactory >	ParticleFactories
typedef vector< Particle >	Particles
typedef vector < ResidueConstFactory >	ResidueConstFactories
typedef vector< ResidueConst >	ResidueConsts
typedef vector< ResidueFactory >	ResidueFactories
typedef vector< Residue >	Residues
typedef vector < RigidParticleConstFactory >	RigidParticleConstFactories
typedef vector < RigidParticleConst >	RigidParticleConsts
typedef vector < RigidParticleFactory >	RigidParticleFactories
typedef vector< RigidParticle >	RigidParticles
typedef vector< ScoreConstFactory >	ScoreConstFactories
typedef vector< ScoreConst >	ScoreConsts
typedef vector< ScoreFactory >	ScoreFactories
typedef vector< Score >	Scores
typedef vector < SegmentConstFactory >	SegmentConstFactories
typedef vector< SegmentConst >	SegmentConsts
typedef vector< SegmentFactory >	SegmentFactories
typedef vector< Segment >	Segments
typedef vector < StaticAliasConstFactory >	StaticAliasConstFactories
typedef vector< StaticAliasConst >	StaticAliasConsts
typedef vector < StaticAliasFactory >	StaticAliasFactories
typedef vector< StaticAlias >	StaticAliass
typedef vector< TypedConstFactory >	TypedConstFactories
typedef vector< TypedConst >	TypedConsts
typedef vector< TypedFactory >	TypedFactories
typedef vector< Typed >	Typeds

Enumerations
enum	Compression { GZIP_COMPRESSION, SLIB_COMPRESSION, NO_COMPRESSION }
enum	NodeSetType { BOND, CUSTOM_SET }
	The types of the nodes. More...
enum	NodeType { REPRESENTATION, GEOMETRY, FEATURE, ALIAS, CUSTOM }
	The types of the nodes. More...

Functions
RMFEXPORT NodeHandle	add_child_alias (NodeHandle parent, NodeConstHandle alias)
RMFEXPORT void	copy_frame (FileConstHandle input, FileHandle output, unsigned int inframe, unsigned int outframe)
RMFEXPORT void	copy_structure (FileConstHandle input, FileHandle output)
RMFEXPORT HDF5File	create_hdf5_file (std::string name)
RMFEXPORT FileHandle	create_rmf_file (std::string path)
RMFEXPORT std::string	get_as_node_name (std::string input)
RMFEXPORT NodeHandles	get_children_resolving_aliases (NodeHandle nh)
RMFEXPORT NodeConstHandles	get_children_resolving_aliases (NodeConstHandle nh)
RMFEXPORT bool	get_equal_frame (FileConstHandle input, FileConstHandle out, unsigned int inframe, unsigned int outframe, bool print_diff=false)
RMFEXPORT bool	get_equal_structure (FileConstHandle input, FileConstHandle output, bool print_diff=false)
RMFEXPORT std::string	get_example_path (std::string file_name)
	Return the path to installed example data for this module.
RMFEXPORT int	get_number_of_open_hdf5_handles (HDF5ConstFile f=HDF5ConstFile())
RMFEXPORT Strings	get_open_hdf5_handle_names (HDF5ConstFile f=HDF5ConstFile())
RMFEXPORT NodeConstHandles	get_particles_by_resolution (NodeConstHandle h, double resolution, int frame=0)
RMFEXPORT std::string	get_set_type_name (NodeSetType t)
RMFEXPORT std::string	get_type_name (NodeType t)
RMFEXPORT HDF5File	open_hdf5_file (std::string name)
RMFEXPORT HDF5ConstFile	open_hdf5_file_read_only (std::string name)
RMFEXPORT FileHandle	open_rmf_file (std::string path)
RMFEXPORT FileConstHandle	open_rmf_file_read_only (std::string path)
void	set_show_hdf5_errors (bool tf)
RMFEXPORT void	show_hierarchy (NodeConstHandle root, bool verbose=false, unsigned int frame=0, std::ostream &out=std::cout)
RMFEXPORT void	show_hierarchy_with_decorators (NodeConstHandle root, bool verbose=false, unsigned int frame=0, std::ostream &out=std::cout)

Typedef Documentation

typedef herr_t(* RMF::HDF5CloseFunction)(hid_t)

The signature for the HDF5 close functions.

typedef vector<HDF5File> RMF::HDF5ConstFiles

typedef vector<HDF5Group> RMF::HDF5ConstGroups

typedef vector<HDF5File> RMF::HDF5Files

typedef vector<HDF5Group> RMF::HDF5Groups

Enumeration Type Documentation

enum RMF::Compression

Data sets can be compressed using one of several algorithms.

enum RMF::NodeSetType

The types of the nodes.

Enumerator:

BOND

A general bond between two or more atoms.

CUSTOM_SET

Arbitrary data that is not standardized.

Programs can use these keys to store any extra data they want to put into the file.

enum RMF::NodeType

The types of the nodes.

Enumerator:

REPRESENTATION	Represent part of a molecule.
GEOMETRY	Store a geometric object.
FEATURE	Store information about some feature of the system. For example, the particles involved in scoring functions and their score can be encoded as feature nodes.
ALIAS	Store a reference to another node. This node should be an alias decorator node and have no other data, at least for now.
CUSTOM	Arbitrary data that is not standardized. Programs can use these keys to store any extra data they want to put into the file.

Function Documentation

RMFEXPORT NodeHandle RMF::add_child_alias	(	NodeHandle	parent,
		NodeConstHandle	alias
	)

Add a child to the node that is an alias to another node.

RMFEXPORT void RMF::copy_frame	(	FileConstHandle	input,
		FileHandle	output,
		unsigned int	inframe,
		unsigned int	outframe
	)

Copy the hierarchy structure and set structure from one rmf file to another.

RMFEXPORT void RMF::copy_structure	(	FileConstHandle	input,
		FileHandle	output
	)

Copy the hierarchy structure and set structure from one rmf file to another.

RMFEXPORT HDF5File RMF::create_hdf5_file ( std::string name )

Create a new hdf5 file, clearing any existing file with the same name if needed. The file cannot already be open.

RMFEXPORT FileHandle RMF::create_rmf_file ( std::string path )

Create an RMF from a file system path.

RMFEXPORT std::string RMF::get_as_node_name ( std::string input )

Node names have to obey certain rules, such as no quotes in the name. This returns a string that has been modified to obey the rules.

String RMF::get_bond_data_data_set_name ( )

Get the name of the data set for storing bonds.

String RMF::get_category_name_data_set_name ( int arity )

Get the name of the data set for storing category names.

RMFEXPORT NodeHandles RMF::get_children_resolving_aliases ( NodeHandle nh )

See get_children_resolving_aliases(NodeConstHandle).

RMFEXPORT NodeConstHandles RMF::get_children_resolving_aliases ( NodeConstHandle nh )

Aliases are nodes that refer to other nodes. Resolving them can result in a graph that is no longer a tree or even a DAG.

String RMF::get_data_data_set_name	(	std::string	category_name,
		int	arity,
		String	type_name,
		bool	per_frame
	)

Get the name of the data set for storing a particular type of data.

RMFEXPORT bool RMF::get_equal_frame	(	FileConstHandle	input,
		FileConstHandle	out,
		unsigned int	inframe,
		unsigned int	outframe,
		bool	print_diff = `false`
	)

Return true of the two have the same structure.

RMFEXPORT bool RMF::get_equal_structure	(	FileConstHandle	input,
		FileConstHandle	output,
		bool	print_diff = `false`
	)

Return true of the two have the same structure.

RMFEXPORT std::string RMF::get_example_path ( std::string file_name )

Return the path to installed example data for this module.

Return the path to an example file from the name.

Each module has its own example directory, so be sure to use the version of this function in the correct module. For example to read the file example_protein.pdb located in the examples directory of the IMP::atom module, do

IMP::atom::read_pdb(IMP::atom::get_example_path("example_protein.pdb", model));

This will ensure that the code works when IMP is installed or used via the tools/imppy.sh script.

String RMF::get_key_list_data_set_name	(	std::string	category_name,
		int	Arity,
		String	type_name,
		bool	per_frame
	)

Get the name of the attribute which lists all the keys of the category.

String RMF::get_node_data_data_set_name ( )

Get the name of the data set storing the data about each node.

String RMF::get_node_name_data_set_name ( )

Get the name of the data set storing the name for each node.

RMFEXPORT int RMF::get_number_of_open_hdf5_handles ( HDF5ConstFile f = HDF5ConstFile() )

For debugging, one can get the number of open hdf5 handles for either one file, or the whole system.

RMFEXPORT Strings RMF::get_open_hdf5_handle_names ( HDF5ConstFile f = HDF5ConstFile() )

For debugging you can get the names of open handles in either one file or the whole process.

RMFEXPORT NodeConstHandles RMF::get_particles_by_resolution	(	NodeConstHandle	h,
		double	resolution,
		int	frame = `0`
	)

Return a list of Particle NodeHandles that forms a slice through the tree and whose radii are as close as possible to the passed resolution.

String RMF::get_set_data_data_set_name ( int arity )

Get the name of the data set for storing bonds.

RMFEXPORT std::string RMF::get_set_type_name ( NodeSetType t )

Return a string version of the type name.

RMFEXPORT std::string RMF::get_type_name ( NodeType t )

Return a string version of the type name.

RMFEXPORT HDF5File RMF::open_hdf5_file ( std::string name )

Open an existing hdf5 file. The file cannot already be open/.

RMFEXPORT HDF5ConstFile RMF::open_hdf5_file_read_only ( std::string name )

Open an existing hdf5 file read only. The file cannot already be open.

RMFEXPORT FileHandle RMF::open_rmf_file ( std::string path )

Open an RMF from a file system path.

RMFEXPORT FileConstHandle RMF::open_rmf_file_read_only ( std::string path )

Open an RMF from a file system path.

void RMF::set_show_hdf5_errors ( bool tf )

Turn on and off printing of hdf5 error messages. They can help in diagnostics, but, for the moment, can only be output to standard error and so are off by default.

RMFEXPORT void RMF::show_hierarchy	(	NodeConstHandle	root,
		bool	verbose = `false`,
		unsigned int	frame = `0`,
		std::ostream &	out = `std::cout`
	)

Print out the hierarchy as an ascii tree.

RMFEXPORT void RMF::show_hierarchy_with_decorators	(	NodeConstHandle	root,
		bool	verbose = `false`,
		unsigned int	frame = `0`,
		std::ostream &	out = `std::cout`
	)

Print out the hierarchy as an ascii tree marking what decorators apply where.

Detailed Description

The RMF library

0 Basics

1 Associations

2 Accelerated I/O

3 Parallel I/O

4 Invariants

5 Other language bindings

6 HDF5 wrappers

Helper programs

Classes

Standard module methods

Batch data access

Helper functions

Data set names

Typedefs

Enumerations

Functions

Typedef Documentation

Enumeration Type Documentation

Function Documentation