Skip to content

Latest commit

 

History

History
3238 lines (2135 loc) · 104 KB

ADMINISTRATION.pod

File metadata and controls

3238 lines (2135 loc) · 104 KB

TITLE

CMap Administration and Data Curation

VERSION

$Revision: 1.47 $

This document is intended to help you understand how to use the data curation tools provided with CMap. There are three tools you will use: the configuration files, the "cmap_admin.pl" command-line interface and the web-base administration interface.

The configuration file is used to create and customize the look of map, feature and evidence types as well as customizing the CMap experience.

The "cmap_admin.pl" program is used for all long-running processes that are not practical to address over the HTTP protocol. It employs a "wizard"-like approach to accomplishing various tasks by asking the curator a series of questions and performing the desired action using the answers provided. cmap_admin.pl can also be driven via command line arguments.

The web admin tool is meant to provide a point-and-click interface for performing the more mundane administrative tasks or for viewing the data.

All of this is discussed in further detail in this document.

STARTING OUT

If you have just installed CMap, then your database is probably completely empty. (Of course, a small but complete dataset is provided in the "data" directory, but the database for your own data will be empty.) So let's start by discussing what you need to do to get a usable installation with your data.

This document assumes that you've already been through all the steps described in the INSTALL.pod document, so you should be able to pull up the web-based admin tool by pointing your browser to "http://your.host.name/cgi-bin/cmap/admin" (or where ever you have installed it). Depending on whether or not you decided to password-protect that URL, you may have to enter a username and password when prompted by your browser.

EDITING CONFIGURATION FILES

The first step to getting your data into CMap is to correctly set up your configuration files. The config files tell CMap how to access your database. Also, each map type, feature type and evidence type of the data that you will be installing needs to be defined in the configuration files.

CMap has multiple configuration files. These files are read out of the "cmap.conf/" directory, likely located in your apache "conf/" directory. There are two types of config files. There is one (and only one) "global.conf" file that holds information applicable to all of the databases. And in that directory, at least one individual configuration file for each database used. Everything that can be customized about CMap is controlled by these files, which are described in detail below.

The configuration files are written in a standard Apache-style configuration file syntax. Comments are defined as anything following a hash sign and are ignored. Options which are grouped together are in angle brackets ("<>"). Standard options are written in a "Name Value" syntax where the two are separated by whitespace or an equal sign. For more information on the syntax, read the POD for Config::General (by executing "perldoc Config::General" on your system).

Each configuration setting is documented in comments. Legal and default values are listed along with a brief description of the option. Except for the "database" and map/feature/evidence_type settings, there is a reasonable default, hard coded value for every configuration setting (so you could actually comment out all of the "optional settings" in the file).

Whenever you make a change to cmap.conf, it will most likely be necessary for you to purge the CMap cache to see the change if the change affects something in the viewer since. This is particularly true if map|feature|evidence type information has been changed because that information is cached along with database search results. Purging the query cache can be done by using the cmap_admin.pl program (explained later in this document).

global.conf

The global.conf contains information which is used by all of the other configurations. You will want to set the default_db value.

  • template_dir

    Automatically set to the path of the directory that contains the html templates.

  • web_document_root_dir

    Automatically set to the absolute path of the document root for the web server.

  • web_cmap_htdocs_dir

    Automatically set the absolute path to the cmap docs directory. This must be in the web_document_root directory.

  • cache_dir

    Automatically set to the path of the directory that contains the temporary image files. This has no relation to the query caching system mentioned earlier (that needs purging).

  • session_dir

    An absolute path to the directory where session files are written.

  • default_db

    Set the name of the database (named in its config file) that will be used as the default.

  • max_img_dir_fullness [ default: 0 ]

    This is the maximum percent the image cache directory will fill before CMap quits writing to it. This is to keep CMap from filling your drive.

    Excluding or setting to 0 will ignore directory size.

  • max_img_dir_size [ default: 0 ]

    Using this option may slow down your requests by as much as 0.4 seconds. This is the maximum size (in bytes) the image cache directory will grow before CMap quits writing to it. This is to keep CMap from filling your drive.

    Excluding or setting to 0 will ignore directory size.

  • purge_img_dir_when_full [ default: 0 ]

    Allow CMap to purge the image directory when it is full.

    Excluding or setting to 0 will prevent CMap from attempting to purge the image directory.

  • file_age_to_purge [ default: 300 ]

    The age (in seconds) of an image file that an image has to be before it can be removed by CMap.

    The default is set to five minutes so that any http requests accessing current images will have a chance to get them. Setting to 0 may cause problems with concurrent requests.

  • max_image_map_objects [ default: 20000 ]

    The maximum number of area boxes (clickable image elements) that CMap will try to print before issuing a warning message and ignoring all area boxes. This can save a browser from crashing. No commas allowed.

  • max_query_cache_size [ default: 26214400 ]

    How large the size limit should be for the query cache in bytes. The value 0 will cause the size to not be limited. This is used in the cmap_reduce_cache_size.pl ("REDUCING QUERY CACHE SIZE") script

INDIVIDUAL CONFIGURATION FILES

CMap supports multiple data sources and multiple customizations for each. This allows a curator to maintain distinct databases and view them all in CMap.

If you have only one database, then you need only one configuration file of this type. Adding another is as simple as creating a new configuration file. Any file that is in the "cmap.conf/" directory that ends in ".conf" will be read as a configuration file.

All of the parameters that can be specified in an individual config file are described here.

The required elements are the "Required General Options" and the "Map, Feature and Evidence Type Information". The other elements all have reasonable defaults and can be ignored until you want to tweak them.

The file "example.conf" (which is installed in the cmap.conf/ directory) provides a good starting point for your configuration. When you are done customizing your config file, make sure you set "is_enabled" to 1.

Required General Options

  • is_enabled

    Set to 1 to enable or 0 to keep this configuration file from being used.

  • <database>

    Contains the database connection parameters. The 'name' field is the name that this config file and the data source will be referred to as. Therefor, it is important to keep it unique.

Map, Feature and Evidence Type Information

Starting with v0.13 map, feature and evidence types (*_types) are defined in the configuration files and not in the database. This means that to add an object of *_type, its type must be defined here in the config file.

The accession id of the *_type is listed in two places for technical reasons. It is important that both places have the same accession id. One is in the initial tag, <map_type X>, where 'X' is the accession. The other is in the 'evidence_type_acc' field.

It is important to note that some of the information is stored in the database when features and map sets are created. Fields like map units and default_rank will not change in the database if you change them only in the config file. This may be something to be addressed in the future.

Common fields.

  • *_type_acc

    The accession id that is used in the database to identify the type. No spaces are allowed. Example: 'maize_marker'

  • *_type

    The full name of that type, such as "Maize Marker".

  • color

    The color of the object.

  • attribute

    Attributes of feature, evidence and map types have been moved out of the database and into the config file.

    Attributes are essentially name, value pairs which describe something about the type. They must be defined in <attribute> tags. Here is an example entry:

    <attribute>
      name Description
      value This is the description of the *_type.
      is_public 1
    </attribute>
    <attribute>
      name History
      value This type was first discovered in the fourth century BC.
    </attribute>

    Notice that is_public is optional. If not given it will default to 1.

  • xref (cross-reference)

    Cross-references of feature, evidence and map types have been moved out of the database and into the config file.

    Cross-references (xrefs) are links from this type to another website. They consist of a name and a URL. If the URL is inside the "cmap/" namespace you can define just the end portion. However if you want to link outside this website, you must include the "http://". Here is an example entry:

    <xref>
      name Map Search
      url map_search
    </xref>
    <xref>
      name google
      url http://www.google.com
    </xref>
  • area_code (map and feature types only)

    CMap allows you to change the URL, the alternate message or specify JavaScript to be included in the area box over the drawings of map and features. The following example makes the map name appear in the status bar of the browser on mouse over. It also changes the alternate message to the map name and makes the url search google.

    $code=sprintf("onMouseOver=\"window.status='%s';return true\"",$map->{'map_name'});
    $alt =sprintf("%s",$map->{'map_name'});
    $url=sprintf("www.google.com/search?q=%s",$map->{'map_name'});

    This is Perl code that lets you change the values of three variables; $code, $alt and $url. $code hold the JavaScript commands that the area will use. $alt holds the alternate message that will pop up when the mouse is hovered over the area box. $url is the URL that the browser will go to when clicking the area box. These are evaluated for each map of this type which enables access to the internal variable $map (or in the case of features, $feature).

    This allows the administrator the freedom to make the page more versatile. Each variable has usable defaults.

    Some useful keys in $map (which are used $map->{'key'}):

    - map_name
    - map_id

    The database identifier, useful for making queries.

    - map_acc

    The map accession id

    - map_type_acc

    The accession id of the map type (of course you already know this)

    - Other Map Keys

    map_start, map_stop, display_order, map_set_id, map_set_acc, map_set_name, map_set_short_name, published_on, shape, width, color, map_units, is_relational_map, species_id, species_acc, species_common_name, species_full_name, map_type_display_order, map_type, epoch_published_on, default_shape default_color default_width

    Some useful keys in $feature (which are used $feature->{'key'}):

    - feature_name
    - feature_id

    The database identifier, useful for making queries.

    - feature_acc

    The feature accession id

    - feature_type_acc

    The accession id of the feature type (of course you already know this)

    - Other Feature Keys

    feature_start, feature_stop, direction, map_id, map_acc, map_name, map_units, feature_type, default_rank, shape color drawing_lane, drawing_priority,

    Since area_code allows you to inject Perl to be executed, the Bio::GMOD::CMap::Data::Generic can be used gather data from the db. To access the module, use the "$self->sql()" method. You can use the object returned to call data retrieval methods. For an list and explanation of the possible methods, execute "perldoc Bio::GMOD::CMap::Data::Generic".

    The following is an example of how to get an xref from the database.

    area_code <<EOF
      my $dbxrefs = $self->sql()->get_xrefs(
        cmap_object => $self,
        object_id   => $feature->{'feature_id'},
        object_type => 'feature',
        xref_name   => 'Chado',
      );
      my $new_url = '';
      if ( @{ $dbxrefs || [] } ) {
        my $t = $self->template;
        $t->process( \$dbxrefs->[0]{'xref_url'},
            { object => $feature }, \$new_url );
      }
      $url = $new_url;
      $code=sprintf("onMouseOver=\"window.status='%s';return true\"",$feature->{'feature_type_acc'});
    EOF
  • required_page_code (map and feature types only)

    This field takes a list of page_code accessions. Page_code is used to insert functions that map/feature JavaScript will use. This is handy if multiple types do the same complex thing. Creating page_code is described later.

  • extra_forms (map and feature types only)

    This field takes a list of extra_form accessions. Extra_forms are used to add form objects to the options menu. These can be used for anything from hidden fields that store data to buttons that submit the page to a different place. Creating extra_forms is described later.

Feature type specific options

  • Shape:

    The shape to use when drawing the feature. Here is a short description of the shapes and what they're best used for:

    - line:

    This a horizontal line just wider than the map's width intended to represent single-point features (e.g., SNPs or markers). If the feature has a length (a defined stop), the line is drawn at the =item *midpoint* of the feature. This shape is the most generic and has the least tendency to look busy on very crowded maps. It is mostly intended to represent single-point features, but is also useful for spanning features when you aren't necessarily concerned with showing the span.

    - in-triangle:

    Like the "line," this is intended to represent single-point features or just the beginning of a spanning feature (e.g., the beginning of an insertion). "In" here means pointing at the map, which may mean pointing right or left depending on the side of the map which features are being drawn.

    - out-triangle:

    Exactly like the "in-triangle," only pointing away from the map. These two glyphs were added for the common representation of insertions and deletions.

    - box:

    Creates an open (unfilled) box that straddles the map. While it's possible to place a box in a lane other than "1" (i.e., there's no code in place to stop you), it really doesn't look very good. This shape was really intended just to represent infrequently occurring features, like a centromere. The boxes get progressively larger as they start to collide and are quite confusing when there are many. Use sparingly.

    - dumbbell:

    This is a spanning shape intended for features with both a start and stop position. The dumbbell is fairly small and floats in mini-lanes within the drawing lane. When a feature with no length (a single-point feature) is drawn with a dumbbell, it renders as a dot (just the circle on the end), which could be confusing to users.

    - span:

    Like half a box, the "span" is a three-sided figure made of straight lines that was intended for spanning features. Looks very odd when applied to single-point feature.

    - direction-arrow:

    Another spanning shape, this is a straight line with an arrowhead pointing in the direction dictated by the 'direction' field. It "floats" like the dumbbell.

    - up-arrow:

    Exactly like the "direction-arrow," only with the arrowhead always points towards the top of the image.

    - down-arrow:

    Exactly like the "direction-arrow," only with the arrowhead always points towards the bottom of the image.

    - double-arrow:

    Like the "up-arrow" and "down-arrow," except with arrowheads on both ends.

    - filled-box:

    Another spanning feature which floats (like the dumbbell), this is a filled-in rectangle (trimmed in black).

    - banding:

    A spanning feature where consecutive banding features alternate in color between black and the chosen color.

    There are banding specific options that can be defined:

    • color (default: red)

      This is the first color that this glyph will oscillate between. This is used in conjunction with the "color2" option. Technically, this isn't banding secific but I thought it was worth a mention here anyway.

    • color2 (default: black)

      This is the second color that this glyph will oscillate between. This is used in conjunction with the "color" option.

    • overlap_color (default: halfway between between color and color2)

      This is the color that overlapping portions will be.

    Also, a feature type with this glyph must have it's own drawing lane.

    - read-depth:

    A spanning feature where the name of the feature dictates how far from the base position a line is drawn. The name MUST be a number. This will create a line graph type of glyph.

    Also, a feature type with this glyph must have it's own drawing lane.

    - heatmap

    A heatmap glyph is a density glyph where density is shown by color. Each color is made by an individual feature and the feature name is the number that this section represents.

    For example, two items are collapsed below.

    ----
      ------------
    |
    V
      -----OOOO---

    This would be represented by three features and written in the import file like this:

    feature_name    feature_start   feature_stop    feature_type_acc
    1               1               5               something_density
    2               6               9               something_density
    1               10              12              something_density

    There are heatmap specific options that should be defined in order for this to be most useful.

    • min_value (default: 0)

      The minimum value that a feature can be named. If it is lower than this, the color will be the minimum color.

    • max_value (default: 100)

      The maximum value that a feature can be named. If it is higher than this, the color will be the maximum color.

    • min_color_value (default: '255,0,0')

      This is the color that the minimum value will have. It is the bottom color of the gradient. It is represented as a string, writen in rgb format ('red,green,blue').

    • max_color_value (default: '0,255,0')

      This is the color that the maximum value will have. It is the top color of the gradient. It is represented as a string, writen in rgb format ('red,green,blue').

    Also, a feature type with this glyph must have it's own drawing lane.

  • feature_modification_code

    The feature_modification_code is similar to the area_code option (see above). It allows you to inject Perl code into CMap when the feature is about to be drawn. Also, like area_code, it can access variables that are in the environment where it is run. To see what this has access to, you can search for "feature_modification_code" in .../path/to/Bio/GMOD/CMap/Drawer/Map.pm.

    The following will change the color of the feature if the name matches a specified pattern.

    feature_modification_code <<EOF
      if ($feature->{'feature_name'} =~ /^sb(\S+)_/){
          $color = 'black';
      }
      elsif ($feature->{'feature_name'} =~ /^os(\S+)_/){
          $color = 'orange';
      }
    EOF
  • glyph_overlap;

    This option was completely unnecessary and has been removed from use.

  • Drawing Lane:

    A positive integer defining the "lane" where the feature will be drawn. The idea was taken from Gbrowse's "tracks," this allows you the ability to segregate feature types into visually distinct vertical areas. Feature types which share the same lane will all be distributed before feature types in the next highest lane. The lower the number for the lane, the closer to the map the feature will be drawn.

    Lanes are not absolute, they are only used to sort the features on a map; therefore, if you have only "Centromeres" and "Markers" in lane #1, on any map with either of those types of features, they will be drawn =item *on* the map; on any map without those types, the feature type with the next highest lane number will be drawn =item *on* the map. The same is true as you move into higher lanes; if the lanes move like "1," "4," "7," you will have only 3 lanes with no gaps.

  • Drawing Priority:

    A positive integer which determines the order in which feature types within the lane are drawn. The lower the number for the priority, the sooner the feature will be drawn in the lane.

  • Feature Default Display (feature_default_display)

    This allows you to choose default display options for individual feature types. This default overrides the global default but is overridden by the feature_default_display defined in the reference map set's map_type.

    Values:
       display:   Display all features of this type
       corr_only: Display only features of this type with displayed correspondences
       ignore:    Completely ignore features of this type.  

Evidence type specific options

A single correspondence can be supported by any number of evidence types, so it is necessary that these be ranked from "1" to N (where lower numbers have precedence over higher). When multiple records support a correspondence, only the highest ranking evidence will be used when determining how to draw the correspondence line. If you intend to use cmap_admin.pl tool to create correspondences for you based on simple name/feature type comparisons, you should create a special evidence to use just for that purpose (e.g., with the name "Automated name-based" or something similar to flag it as a machine-created/non-curated correspondence). Evidence type color will only be used when NOT aggregating.

  • Rank:

    The rank of the evidence relative to the others

  • Line Type (line_type):

    How lines representing this evidence type will be drawn.

    Values:
       direct:   Line directly from feature to feature.
       indirect: Line that starts perpendicular to the map for a short distance
                 before traveling to the other feature.
       ribbon:   A ribbon is drawn that spans the top and bottom of each feature. 
  • Evidence Default Display (evidence_default_display)

    This allows you to choose default display options for individual evidence types. This default overrides the global default.

    You cannot default to the greater than or less than score options.

    Values:
       display:      Display all correspondences with evidence of this type
       ignore:       Completely ignore the evidence of this type when considering correspondences.  
  • <aggregated_correspondence_colors>

    This option dictates the color of aggregated correspondence lines for this evidence type. It is defined the same way as aggregated_correspondence_colors is in the main options section of "INDIVIDUAL CONFIGURATION FILES".

    These colors will only be used if the aggregate correspondences are split based on evidence type. Meaning each evidence type will have it's own aggregated correspondence. It is suggested to use primary colors for each different type, so they can be distinguished.

    <aggregated_correspondence_colors>
       1    lightblue
       20   blue
       0    darkblue
    </aggregated_correspondence_colors>

Map type specific options

  • Unit Granularity (unit_granularity)

    Define the smallest unit that the map can be broken into. For example, a sequence has a unit_granularity of 1 because 1 base pair is the smallest possible unit. A genetic map however, might have a granularity of .01.

    The unit_granularity must be defined for each map_type for a configuration file to be considered valid by the validator. However, CMap will work without it.

  • Is Relational Map (is_relational_map):

    Are maps in this set only relational maps? Set this to 1 if there are a lot of maps in the set and they don't need to selected individually. This keeps the render times down and makes for smaller menus.

  • Map Units (map_units):

    The units of measurement for this type of map. This text will display at the bottom of non-relational maps, so it's best to keep this short, like "cM" for "centimorgans."

  • Display Order (display_order):

    Like the same field in the "species" table, this determines the order that maps of this type will be displayed, e.g., on Gramene it is preferred that "Sequence" maps be listed before "Genetic" maps which should come before "Physical" maps.

  • Shape (shape):

    This determines how maps of this type will be drawn. You can choose to make maps of each type visually distinct by specifying different shapes and colors. There are three choices for shape: box, dumbbell and I-beam. A box will be trimmed in black and will be filled with the color of your choice (or a reasonable default). A dumbbell or an I-beam will be rendered only in the map's color (no trim).

  • Width (width):

    How wide a map will be drawn.

  • Feature Default Display (<feature_default_display>)

    This allows you to choose default display options for individual feature types when this maps of this map_type are used as reference maps. This default overrides the both the global default and the default set in the feature_type.

    The values are defined as name, value pairs, where the name is the feature_type accession and the value is the default value for that feature_type. They must be defined in <feature_default_display> tags. Here is an example entry:

    <feature_default_display>
      marker display
      read corr_only
    </feature_default_display>
    
     Values:
     display:   Display all features of this type
     corr_only: Display only features of this type with displayed correspondences
     ignore:    Completely ignore features of this type.  

Menu Defaults

Map Menu Options

  • stack_maps [ default: 0 ]

    Place the reference maps vertically rather than side by side.

  • menu_bgcolor [ default: white ]

    The background color of some areas of the menu

    Values: color

  • menu_bgcolor_tint [ default: lightgrey ]

    The background color of tinted areas of the menu

    Values: color

  • menu_ref_bgcolor [ default: aqua ]

    The background color of some of the reference map areas of the menu

    Values: color

  • menu_ref_bgcolor_tint [ default: lightblue ]

    The background color of tinted reference map areas of the menu

    Values: color

Feature Menu Options

  • label_features [ default: all ]

    Which features to show by default

    Values: none, landmarks, all

  • collapse_features [ default: 1 ]

    Collapse features glyphs into one glyph if they overlap

    Values: 
       0:  don't collapse
       1:  collapse
  • feature_default_display [ default: display ]

    Dictates the default choice of the feature type display menu. This is overridden by defaults in the map_type (of the reference map set) or the feature_type definitions.

    Values:
       display:   Display all features of this type
       corr_only: Display only features of this type with displayed correspondences
       ignore:    Completely ignore features of this type.  

Correspondence Menu Options

  • aggregate_correspondences [ default: 1 ]

    Set default, aggregate correspondences into one or two lines.

    Values: 
       0:  don't aggregate 
       1:  one line 
       2:  two lines
  • evidence_default_display [ default: display ]

    Dictates the default choice of the evidence type display menu. You cannot default to the greater than or less than score options.

    This default is overridden by the any individually defined defaults.

    Values:
       display:      Display all correspondences with evidence of this type
       ignore:       Completely ignore the evidence of this type when considering correspondences.  
  • show_intraslot_correspondences [ default: 0 ]

    Display correspondences between maps in the same slot (map_set).

  • corrs_to_map [ default: 0 ]

    This determines where the correspondence lines are terminated.

    Values:
       0 : Lines connect features
       1 : Lines connect maps
  • split_agg_evespondences [ default: 0 ]

    If multiple correspondence types are present, aggregate them separately from each other.

Display Menu Options

  • font_size [ default: small ]

    How large to make the font

    Values: small, medium, large

  • image_size [ default: small ]

    How large to make the image

    Values: small, medium, large

  • image_type [ default: png ]

    The output type of the image, legal value are whatever image libraries installed

    Values: png, jpeg, svg and maybe gif

  • clean_view [ default: 0 ]

    If set to 1, the control buttons on the image are removed. This is useful for taking images for publication.

  • hide_legend [ default: 0 ]

    If set to 1, the legend on the image is removed. This is useful for taking images for publication.

  • dotplot_ps (Size of Dot Plot Pixels) [ default: 1 ]

    Sets the size of the lines in the dotplot view. Must be a positive integer.

Advanced Menu Options

  • scale_maps [ default: 1 ]

    Allow scaleable (set by scalable) maps to be drawn in scale

    Values: 
       0:  don't scale
       1:  scale
  • <scalable> [ Optional ]

    Dictates which map units can be used to scale maps to each other. Maps with units described here will be scaled against maps with the same units.

  • <scale_conversion>

    If you want to scale maps with different map units, the conversion factor must be specified here.

    For an example (a completely nonsensical example), lets pretend that a centimorgan (cM) is 100 base pair (bp). The config would look something like the following, which is read 'One centimorgan is the size of 100 base pairs'.

    <scale_conversion>
      <cM>
        bp 100
      </cM>
    </scale_conversion>

    Since the directionality doesn't matter, this is the same as:

    <scale_conversion>
      <bp>
        cM .01
      </bp>
    </scale_conversion>

    There is no reason to define the conversion in both directions since only one factor will be used and it could create inconsistencies if one direction was different from the other.

    Also, multiple conversions can be defined as in this (nonsensical) example.

    <scale_conversion>
      <bp>
        cM .01
        bands .1
      </bp>
      <cM>
        bands 10
      </cM>
    </scale_conversion>

    It is appropriate to describe all relationships as CMap won't necessarily figure out the relationship between a band and a centimorgan transitively (but it might depending on the order of the map sets). Therefor, to avoid inconsistencies, please make sure to define all relationships (and define them correctly).

  • omit_area_boxes [ default: 0 ]

    This determines which area boxes are rendered. The area boxes create clickable areas on the image. Rendering all the boxes gives the most functionality but can be slow if there are a lot of features. The option to omit feature area boxes can help speed things up while leaving the navigation buttons. There is also an option to ignore all of the area boxes, which is not recommended.

    Values:
       0 : render all of the area boxes
       1 : omit the feature area boxes 
       2 : omits all area boxes
  • comp_menu_order [ default: 'display_order' ]

    Dictates the default value of how the comparison maps will be ordered in the menu select box. The choices are 'display_order' and 'corrs'. 'display_order' is the classical order based on the maps display order and their names. Specifying 'corrs' orders the maps by the number of correspondences.

  • ignore_image_map_sanity [ default: 0 ]

    A sanity check on the size of the image map (number of clickable objects) is performed unless this is set to 1. If the sanity check fails, the image will not be clickable and a warning message will be displayed.

Presentation Options

  • min_map_pixel_height [ default: 20 ]

    The smallest any map can be drawn, in pixels

    Values: any positive integer (within reason)

  • min_tick_distance [ default: 40 ]

    Set the minimum number of pixels between tick marks on map.

  • feature_color [ default: black ]

    Default color of a feature if the feature type's "color" is "Default"

    Values: see background_color

  • feature_highlight_bg_color [ default: red ]

    Color of box around a highlighted feature

    Values: see background_color

  • feature_highlight_fg_color [ default: yellow ]

    Color of background behind a highlighted feature

    Values: see background_color

  • connecting_line_color [ default: lightblue ]

    The color of the line connecting feature correspondences

    Values: see background_color

  • feature_correspondence_color [ default: red ]

    Color of a feature label when it has a correspondence. Comment out to use the feature's own color

    Values: see background_color

  • aggregated_correspondence_colors [ Optional ]

    This option dictates the color of aggregated correspondence lines based on the number of correspondences between the maps. The default is black.

    The format is "upper_bound color" where upper_bound is the cap that this color is used for. In the following example, from 0-1 corr, the line will be light grey. If there are 2 corr, the line will be blue. From 3-5 (inclusive), the line will be purple. Anything ABOVE 20 will be black, which is denoted by an upper_bound of 0 (in this case it means infinity).

    <aggregated_correspondence_colors>
       1    lightgrey
       2   blue
       5   purple
       20  red
       0    black
    </aggregated_correspondence_colors>
  • background_color [ default: lightgoldenrodyellow ]

    The background color of the map image

    Values: any color found in the COLORS array of Bio::GMOD::CMap::Constants To view this file, either look in the "lib" directory of the original CMap source directory, or, if you have the very handy "pmtools" installed on your system, type "pmcat Bio::GMOD::CMap::Constants" on your command-line.

  • slot_background_color [ default: beige ]

    The colors of the slot background and border.

    Values: see background_color

  • slot_border_color [ default: khaki ]

    The colors of the slot border.

    Values: see background_color

  • user_pref_cookie_name [ default: CMAP_USER_PREF ]

    Name of the cookie holding user preferences

  • cookie_domain [ Optional ]

  • stylesheet [ default: /cmap/cmap.css ]

    The css formatted stylesheet to use.

  • feature_type_details_url [ default: feature_type_info?feature_type ]

    This is the base url for the page that describes the feature type.

  • evidence_type_details_url [ default: evidence_type_info?evidence_type ]

    This is the base url for the page that describes the evidence type.

JavaScript options

These options are only used to allow complex JavaScript to be specified in the config document. These can be used in conjunction with the area_code of feature_types and map_types.

  • page_code

    Each page_code has one field called "page_code" that takes in script information in plain text. It is not as complicated as the area_code in map and feature types. The example is JavaScript but it really can be anything recognized by your browser.

    This can be used to specify functions that the individual maps or features need to use. When a feature or map type is used that requests a page_code object, the code is inserted at the top of the page.

  • extra_form

    Similar to page_code, one or more extra_forms can be called from map or feature types. Each extra_form has one field called 'extra_form' which takes standard html and will insert it into the main CMap viewer form. These can be used for anything from hidden fields that store data to buttons that submit the page to a different place.

Configuration Defined Buttons

CMap allows the creation of buttons to modify options in the CMap viewer menu. Conditions can be set that would need to be met before the button will appear.

A simple example would be to add a "For Publication" button that would set the "clean_view" option and would only appear when the "clean_view" option was off.

The impitus for adding this feature was for creating a button that would toggle between an overview style view and a more detailed view. Clicking the "Overview" button would set the feature display options to "corr_only" of all but a select set of feature types which would have the effect of de-cluttering the view. A "Detailed" button could set all the feature display options to always "display".

Configuration Structure

All buttons are enclosed in an <additional_buttons> tag.

The following is the configuration for the "clean_view" buttons mentioned above.

<additional_buttons>
    <button>
        text For Publication
        <if>
            clean_view 0
        </if>
        <set>
            clean_view 1
        </set>
    </button>
    <button>
        text Return Navigation Buttons
        <if>
            clean_view 1
        </if>
        <set>
            clean_view 0
        </set>
    </button>
</additional_buttons>

Note that there are two buttons, one for when "clean_view" is set and one for when it is not. They work together.

Each <button> is it's own element with the following fields.

  • text

    The text written on the button. This should be short, otherwise there will be some ugly buttons.

  • if (Optional)

    This section contains one or more tests for menu items. All options must match for the button to be displayed.

    The individual options are described later in the Testable Parameters section.

  • if_not (Optional)

    This section contains one or more tests for menu items. All options must not match for the button to be displayed.

    The following is an alternate configuration for the "For Publication" button.

    <button>
        text For Publication
        <if_not>
            clean_view 1
        </if_not>
        <set>
            clean_view 1
        </set>
    </button>

    The individual options are described later in the Testable Parameters section.

  • set

    This is where the action of the button is defined. It will set the value of the defined parameters to the value supplied. The defined parameter must be the name of a menu option. If you are unsure of the menu option name, simpley view source on a CMap page and look for the option that you wish to set.

Testable Parameters

- Boolean Parameters

These parameters can be tested using 1 or 0 (or blank).

highlight
collapse_features
scale_maps
stack_maps
omit_area_boxes
show_intraslot_corr
split_agg_ev
clean_view
corrs_to_map
ignore_image_map_sanity
dotplot
- String Parameters

These parameters are a little more than boolean. They may be numbers (for isntance, the radio button" aggregate" values are 0,1 or 2). View page source of the CMap viewer to see the options of radio buttons.

prev_ref_species_acc
prev_ref_map_set_acc
ref_species_acc
ref_map_set_acc
image_type
label_features
aggregate
comp_menu_order
data_source
ref_map_start
ref_map_stop
font_size
pixel_height
dotplot_ps
- Feature and Evidence Type Parameters

These parameters are a little different. To query if a feature type is displayed, that feature type must be included as a display_feature_type.

display_feature_type feature_type_acc
corr_only_feature_type feature_type_acc
ignored_feature_type feature_type_acc

included_evidence_type evidence_type_acc
ignored_evidence_type evidence_type_acc
less_evidence_type evidence_type_acc
greater_evidence_type evidence_type_acc

An example button:

<button>
    text No Marker
    <if>
        display_feature_type marker
    </if>
    <set>
        ft_marker 0
    </set>
</button>

Miscellaneous Options

  • disable_cache [ default: 0 ]

    Set to 1 to disable the query caching feature. This is good for speed testing or debugging.

  • feature_search_field [ default: feature_name ]

    Set the default search field.

    Values: feature_name, feature_aid

  • max_search_pages [ default: 10 ]

    How many pages of results to show in searches. Set to "0" (or a negative number) or comment out to disable

    Values: any positive integer (within reason)

  • max_child_elements [ default: 25 ]

    The maximum number of elements that can appear on a page (like in search results). Set to "0" or a negative number or comment out to disable

    Values: any positive integer (within reason)

  • object_plugin [ optional ]

    See object-plugins.pod for further details.

  • page_object [ optional ]

    Returns the "page" object. In the original Gramene project (http://www.gramene.org/), there is a Perl object that handles inserting appropriate page headers and footers. This object can be passed into the templates to print out these elements as necessary.

    If you wish to have something like this, I'm sure you can glean the basic ideas from Lincoln's "GramenePage.pm" module. Then define the "page_object" in your cmap.conf. Otherwise, just don't worry about it, and this method will never return anything.

  • template_toolkit_flags [ optional ]

    This option sets flags during the creation of the Template Toolkit object allowing for more powerful use of Template Toolkit. These flags are passed into the Template->new() method.

    It will only take strings as values (no complicated objects).

    <template_toolkit_flags>
      EVAL_PERL 1
      SOMETHING else
    </template_toolkit_flags>

Title and Intro Text Options

Titles and Introductory texts for various pages.

  • cmap_title [ default: CMap ]

  • cmap_home_intro [ Optional ]

  • map_viewer_intro [ Optional ]

  • species_info_intro [ Optional ]

  • map_set_info_intro [ Optional ]

  • feature_type_info_intro [ Optional ]

  • map_type_info_intro [ Optional ]

  • evidence_type_info_intro [ Optional ]

  • feature_search_intro [ Optional ]

  • matrix_title [ Optional ]

  • matrix_intro [ Optional ]

  • link_info_intro [ Optional ]

  • map_titles [ default: species_common_name map_set_short_name map_name ]

    The list of information to display in the map title box.

Administration Options

  • add_name_correspondence [ Optional ]

    When making name-based correspondences, the default behavior is to allow correspondences between features of the *exact* same type. If you wish to expand the feature types allowed when making name-based correspondences, then add their accession IDs to this section. The relationships will be reciprocal, so if you say "foo bar" then correspondences from features with the accession ID "foo" will be allowed to those with the accession ID "bar" and vice versa. Separate feature types with spaces. Assert all equivalent feature types on one line, e.g., "foo bar baz."

  • disallow_name_correspondence [ Optional ]

    List of feature types that cannot have name correspondences with features of the SAME type.

  • make_corr_feature_divisor [ default: 1 ]

    In response to database problems making correspondences with really large maps, the features on a map can be divided (based on feature_id) to decrease the number of features that the database has to handle and one time. If you are not having this problem, it will not help you at all.

Map Set Creation Defaults

These are the default settings for when map sets are created.

  • map_color [ default: lightgrey ]

    Default color of a map if the "color" field of both the map type and the map set are "Default"

    Values: see background_color

  • map_shape [ default: box ]

    Default shape of a map if the "shape" field of both the map type and the map set are "Default"

    Values: any shape found in the VALID hash under map_shape in Bio::GMOD::CMap::Constants

  • map_width [ default: 8 ]

    Default width of the maps

    Value: any number from 1 to 10

INCLUDING OTHER CONFIGURATION FILES

If there is a section of the individual configuration files that is repeated in multiple files, that section can be placed in a separate file. That file can then be included in each configuration file.

<<include foo_type_info.cfg>>

It is recommended that any included files be named with a ".cfg" extension. Do no name it with a ".conf" extension, otherwise it will be read as a separate config file.

VALIDATING CONFIGURATION FILES

You can use the cmap_validate_config.pl ("VALIDATE CONFIG FILES") to test your individual config files (not global.conf yet) and make sure that they are valid. This can save you a lot of headaches dealing with configuration problems. See the section on "VALIDATE CONFIG FILES" for more information.

CONFIGURING FOR SPEED AND CLARITY

There are a few configuration options that can be used to increase the speed and clarity of CMap. This is a collection of those options. The details are described above.

All of these can be overridden by the user.

  • aggregate_correspondences

    Setting aggregate_correspondences to 1 (or 2) will cause the correspondences in slots with multiple maps to be aggregated. This reduces the clutter if there are many correspondences to draw.

  • collapse_features

    Collapses the features that would be identically drawn into one feature. This is especially important when there are a lot of features on a map and can reduce the drawing speed. It also can make the image more clear.

  • feature_default_display

    Setting feature_default_display to "corr_only" will keep from drawing features that do not have correspondences displayed. This can give a huge savings when there are a lot of features on a map.

    feature_default_display can be overridden by the feature_type configs and by the user. If there are some feature types that should be displayed initially, you can set an individual feature_default_display value in the feature_type configuration.

  • omit_area_boxes

    Setting omit_area_boxes to 1 will keep the features from being clickable. This can have major savings in transmission and render time if there are a lot of features. The map controls will still be clickable as normal.

    Setting omit_area_boxes to 2 will render the whole image unclickable. This could save time but makes CMap harder to use since it relies on clicking the image for a lot of navigation. If you do choose to go this route, CMap will set clean_view to 1 automatically to keep from drawing the buttons that were rendered useless.

WEB-ADMIN TOOL

The home page for the web-based CMap administration tool has links for all the different actions you can take:

  • Create New Map Set

  • View Map Sets

  • View Species

  • View Map Types

  • View Feature Types

  • View Correspondence Evidence Types

  • View Cross-References

  • Search for a Feature

  • View Color Palette

  • Exit to Map Viewer

Each link is self-explanatory. In addition to the above links, you will also see the current data source to which you are connected. You may configure CMap to connect to different distinct data sources by having multiple configuration files; see the section "EDITING CONFIGURATION FILES" above for more information on this. If you have configured more than one data source, you will also see a drop-down control allowing you to connect to a different data source.

On each "View" page, you have access to edit and delete the information there (except for map, feature and evidence type information which are defined in the config file). Each page will be discussed in detail later. For the moment, let's just focus on what you need to import your first data set. To do that, you'll need to first set up the species.

CREATING AND IMPORTING DATA

IMPORTING GFF DATA

Starting with CMap version 1.01, CMap is able to import data in the GFF3 file format.

Using special CMap specific extensions, all the CMap data can be imported in one file. This allows a whole data base worth of CMap data to be imported in one step (rather than the multiple previously required which are described below).

Alternatively, regular GFF3 (as used by GBrowse) can be imported if a map set is specified.

The format is described in detail in the Bio::DB::SeqFeature::Store::cmap module. It can be accessed by running the command:

$ perldoc Bio::DB::SeqFeature::Store::cmap

The GFF files can be imported using the cmap_admin.pl script, either using the menu system or by using the command line interface (examples follow).

The first example is using a gff file "cmap_style.gff" that has the CMap extensions to define which species and map sets the data belongs.

$ cmap_admin.pl -d DATASOURCE --action import_gff cmap_style.gff

This second example defines the map set accession to import the data into. However, if a map set is defined in the file, the map set in the file will be used.

$ cmap_admin.pl -d DATASOURCE --action import_gff --map_set_acc MAP_SET_ACCESSION gbrowse_style.gff

Note: In order for GFF importing to work correcly, it requires bioperl-live downloaded after June 7th, 2008.

EXAMPLE GFF IMPORT

A sample CMap GFF3 file, "test_data.gff", is located in the data directory of the distribution. This can be imported into the demo database (which can be created using the "./Build demo" command).

$ cmap_admin.pl -d CMAP_DEMO --action import_gff data/test_data.gff

SETTING UP SPECIES

Click on "View Species" from the home page of the web-admin tool. As you currently have no species to view, you should see the message "No species to show." To add a new species, click on the "Create New Species" link at the top of the page. You should now see a page entitled "Species Create" with a form for the following fields:

  • Acc. ID (opt.):

    The accession ID

  • Common Name: This is the non-scientific name of the species. This field will be used most often as a prefix to the map names. If you plan on having more than one species that could go by the same common name (e.g., two species of "wheat"), you should differentiate them here, too.

  • Full Name:

    This should be the full genus and species name for the species

  • Display Order (opt.):

    This is the order in which you'd like this species to appear in listings, e.g., on Gramene we prefer "rice" to appear before other species as it is our model organism

Note: All the accession id columns in the CMap tables act the same. They are all character fields, so they will accept any combination of numbers and letters you care to use. Please don't use spaces or characters outside the ranges "a-z," "A-Z," "0-9" or dashes ("-") as this will likely only cause you headaches. It is also not necessary to explicitly assign any accession IDs. While they *are* required by the database, there is code in place to ensure that the accession ID is set to the primary ID of the record if the accession ID is empty. Once your accession IDs have been established and publicized, they should never change.

Also, it is best to avoid strictly numeric accession ids since the automatic accessions are numeric and this can cause conflicts.

The fields marked "(opt.)" do not require you to enter a value. If one is really required for the database (e.g., the "accession_id"), then a reasonable default will be provided (e.g., the primary key value for accession IDs). When you are done, hit the "Submit" button. If there are errors, they will be reported to you and you will have to correct them before submitting again. If there are no errors, your entry will be accepted and you will be returned to the "Species View" page.

Note: On all the pages with clickable column headers, clicking on the column names will resort the data by the column.

If you are unhappy with any of the data you see, you can click on the "Edit" link of the record that displeases you and correct the faults. If you have created an unnecessary species, you can delete it by clicking on the "Delete" link. After confirming that you really wish to delete the species, it will be *permanently* removed from the database.

Note: There is no "undo" function for deletes, so be sure whenever you decide to remove an object from the database that you really mean it. When an object has other objects that rely on it (e.g., species records are linked to map sets), you will not be allowed to remove the object until all dependencies to it are removed (e.g., no more map sets use the species you want to remove).

Once you've set up all the species you wish to have in your database, click on the "Home" link in the upper-left corner to return to the web-admin home page.

Note: Throughout the admin interface, the "create" and "edit" pages for any object (e.g., species, map types, map sets, etc.) have the same fields in the same order with the same restriction. If this document only mentions the "create" or "edit" page for an object, rest assured that the complementary page works the same as the one described.

CREATING A NEW MAP SET

Once you have species set up and map types defined, you are ready to create a new map set. Everything in CMap is designed to be generic, so a "map set" is simply a collection of maps. What you group together is entirely up to you. On Gramene, map sets tend to correspond to published studies of organisms and contain maps that represent chromosomes, linkage groups, FPC contigs, and such. As the database design allows only one species and one map type to be linked to a map set, it is best to keep these narrowly defined.

USING THE WEB ADMIN TOOL TO CREATE A MAP SET

To create a new map set, click on the "Create New Map Set" link from the admin home page or from the "Map Sets View" page. If you have failed to set up species or map types, you will be prompted to do so before continuing. You will see the following fields:

  • Acc. ID (opt.):

    The accession ID

  • Map Set Name:

    This is the canonical name for the map set. It is expected that this name will be quite long and informative, e.g., "Cornell Avena atlantica/A. hirtula RFLP 1995," but it doesn't have to be. Given this expectation, the name that appears in this fields won't actually show up all that often. Mostly, it will show up only on the "map set info" page.

  • Short Name:

    This is the short version of the map set name, e.g., "Cornell Diploid 1995" for the name above. This name is what will be used in all the drop-down boxes and for the map titles on the images and in the matrix. The field is limited to 30 characters.

  • Species:

    This is the one (and only one) species with which the map set is associated.

  • Map Type:

    The type of maps that are contained in this set. If there is nothing in the menu, please make sure that map_types have been defined in the configuration file in use.

  • Published On:

    The date on which the study was published. This is used to sort the map sets. Most date formats will be understood and converted into the proper format for your database.

  • Display Order:

    The order in which you'd prefer this map set to sort above others. Used before other sorting criteria.

  • Is Enabled:

    Determines whether or not this map set will appear to end users. This is intended as a convenient way to temporarily hide some data without having to remove it entirely from the database.

  • Is Relational Only Map

    This determines whether or not the map of this map set can serve as reference maps, i.e., as a starting point in a user's search, either via the main map viewer or the feature search. Relational maps CANNOT serve as a starting point. The default is 0.

  • Shape (opt.):

    The shape that maps of this set will be drawn.

  • Width (opt.):

    The width that maps of this set will be drawn.

  • Color (opt.):

    The color that maps of this set will be drawn.

USING THE cmap_admin.pl TOOL TO CREATE A MAP SET

You can also create a new map set by using the "cmap_admin.pl" tool. There are only a few functions that exists in both tools, and this one was only added to cmap_admin.pl in order to make importing new data sets more convenient. You can only specify the species, map type, long and short names, and accession ID for the map set. Everything else about the map must be edited using the web admin tool.

To create a new map set with cmap_admin.pl, start the script and choose the "Create new map set" option. Answer the questions appropriately and confirm your decision. If you see no errors, the map set was successfully created.

Note: When using cmap_admin.pl, questions which have only one choice are automatically answered by the tool. In the above example, if you had only one species in the database, cmap_admin.pl will not ask you which species to associate with the new map set. There can only be one answer, so it answers the question automatically.

VIEWING NEW MAP SET

Once you have successfully created a map set with the web admin tool, you will be taken to the view of that map set. You should see the data that you entered and that the set currently has no maps associated with it. You can either create each map for the map set individually or you can import the data for the map set. Most likely, you will want to do the latter, so that is the next section.

IMPORTING MAP DATA

Using the cmap_admin.pl, you can import a tab-delimited file containing the data for a map set with the following fields and data types:

  • map_name (character(64)) [REQUIRED]:

    The name of the map (AKA "linkage group," "chromosome," etc.).

  • map_acc (character(20)):

    The accession ID of the map.

  • map_display_order (integer):

    The order in which to display the map.

  • map_start (double(8,2)):

    The start position of the map.

  • map_stop (double(8,2)):

    The stop position of the map.

  • feature_acc (character(20)):

    The accession ID of the feature.

  • feature_name (character(32)) [REQUIRED]:

    The name of the feature.

  • feature_alt_name (character(32)) [DEPRECATED]:

    The alternate name of the feature; deprecated in lieu of "feature_aliases" but still supported.

  • feature_aliases (character(255) each):

    Any number of comma-delimited aliases, each of which can be a maximum of 255 characters

  • feature_start (double(8,2)) [REQUIRED]:

    The starting position of the feature on the map.

  • feature_stop (double(8,2)):

    The ending position of the feature on the map.

  • feature_direction (double(8,2)) :

    The direction of the feature on the map, 1 or -1. 1 is default.

  • feature_type_acc (character(32)) [REQUIRED]:

    The accession id of the feature's type.

  • feature_dbxref_name (character(32)) [DEPRECATED]:

    The name of the database cross reference; deprecated in favor of "feature_attributes."

  • feature_dbxref_url (character(200)) [DEPRECATED]:

    The URL of the database cross reference; deprecated in favor of "feature_attributes."

  • feature_attributes:

    A semi-colon-delimited list of feature attributes.

  • is_landmark ("1" or "0"):

    Whether or not the feature should be marked as a landmark feature (allowing it to be labeled when a user chooses "Landmarks" for the "Label Features" option).

For a more thorough treatment of these fields, read the "import_tab" section of "perldoc Bio::GMOD::CMap::Admin::Import."

The first line of the file should be the tab-separated names of the fields in whatever order you're supplying them (the order of fields is not important). You should use the above names for the fields, but you can use spaces and capitalization for the column names, if you like, as spaces will be converted to underscores and the names lowercased (e.g. "Feature Alt Name" will become "feature_alt_name").

If the fields "map_start" and "map_stop" are not supplied, then the start and stop positions of the map will be determined after importing all the features by selecting the MIN and MAX start and stop positions from the "cmap_feature" table.

Use the "cmap_admin.pl" script to bring in your correctly formatted data. Run the script, optionally passing your data file as an argument, like so:

$ cmap_admin.pl my_groovy_maps.dat

If you pass a file as an argument, you will be asked to confirm that that is the file you want to use. If you answer "no" or did not pass a file, then you will be asked to locate the file containing your data. Type in the path to the file (noticing that you can use tab completion), or type "q" to exit file selection and return to the "Main Menu." Once you've found your file, you'll need to tell the tool which map set the data corresponds to. First choose the map type and then the species of the map set, then the map set itself. Lastly, you will need to confirm your choices. If all goes well, you should see a lot of lines fly by giving you the step-by-step progress, the message "Done," and then you will be returned to the "Main Menu." A complete log of the actions taken by the script will be stored in a file called "cmap_admin_log.X" (where "X" is an incrementing number). See the docs on cmap_admin.pl (by typing "perldoc cmap_admin.pl") for more info.

When you import data for an map set that already has data, all existing maps and features with the same name as maps and features in your data will be updated. You also have the option of deleting any data that isn't updated. If you choose this "overwrite" option, any of the pre-existing maps or features that aren't updated will be deleted as it will be assumed that they are no longer present in the dataset.

While the import process is running, it may encounter feature types in your data which do not exist in the database. If this happens, the program will die right there and you will be left to the task of figuring out what was the last data inserted, defining the feature type in the config file and re-running the remaining data.

When cmap_admin.pl has successfully finished importing your map data, you will be returned to the "Main Menu." From here, choose to "Quit" as the rest of the functionality will be covered later.

IMPORTING XML DATA (EXPERIMENTAL)

As of 0.10, CMap exports and imports data in XML format. The standard tab-delimited format is very convenient and easy to generate, but it's also difficult to indicate hierarchical relationships among data. As such, some experimental has been added to export the concept of "objects" from the database. These objects will contain all the information within them necessary to duplicate themselves entirely in another CMap database. This code is functional but still marked "experimental" as it appears to be very slow when exporting very large map sets.

To try this feature, choose "Export data" and then "Database objects." Follow the directions from there. Then try importing the data (into another database, of course) using the "Import data" option and then "Import CMap objects." Database objects are only created on import at this time; there is no updating of existing objects, so be careful!

MAKING CORRESPONDENCES

Once you have more than one set of maps in CMap, you can use it for what it was designed: showing comparisons. To do this, you must first create the correspondences between the features on the maps. Before you can do this, you'll need to establish the evidence types that can be used to support the correspondences. These are defined in the configuration file. For how to create evidence types, see "EDITING CONFIGURATION FILES" above.

CREATING CORRESPONDENCE RECORDS

Once you have evidence types, you can create the correspondence records. It might be helpful to you to inspect "cmap-schema.png" and "cmap-schema-graph.png" images in the "docs" directory that visualize the CMap tables and their relationships. (Also included is "cmap-schema-desc.html," a breakdown of the tables into HTML tables for easy viewing.) The tables involved are:

  • cmap_feature

  • cmap_feature_correspondence

  • cmap_correspondence_lookup

  • cmap_correspondence_evidence

For descriptions of these tables, look at the "CODE_OVERVIEW.pod" document in the "docs" directory.

There are three ways to create correspondences:

1 Create each by hand using the web admin tool.

To do this, locate a feature on a map by:

A) "View" a map set, then "view" a map, then "view" the feature

B) Click on "Search for a Feature" and find the feature by name

Once you have located a feature and have navigated to the "View Feature" page, click on the link to "Add Correspondence." This will take you to the "Feature Correspondence Create" page where you can search for the other feature in the relationship. When you have located both features, you will be presented a form with the following fields:

  • Evidence Type:

    The evidence type which supports this correspondence. If you wish to add more, you can do so after creating the relationship.

  • Acc. ID (opt.):

    The accession ID

  • Is Enabled:

    You can hide correspondences by setting this field to "No." The intention was that a curator could permanently negate a certain correspondence, e.g., while running the automated name-based comparison, a feature name "ABD104" is found on two maps so the correspondence is created; however the curator knows that this is an invalid correspondence, so, rather than deleted the correspondence, he sets "Is Enabled" to "No" thereby preventing the correspondence from being created again in the future (i.e., additional evidence types may be added to the correspondence, but it won't be enabled -- only the curator can re-enable a correspondence using the web admin tool).

When you have finished, click the button entitled "Create Correspondence." If the correspondence is successfully created, you will be taken to the "View Feature Correspondence" page showing you the two features, their respective maps and types, and the evidences supporting this relationship. You can add more evidence types by clicking the link "Add Evidence." You can also edit and delete existing evidences by clicking the appropriate links to the right of them. If you choose to "Edit" a correspondence, you will be presented a form with the following fields:

  • Acc. ID (opt.):

    The accession ID.

  • Evidence Type:

    The evidence type.

  • Score (opt.):

    A floating-point number representing some value associated with the correspondence, e.g., a BLAST similarity score or an e-value.

2 Automatic name-based correspondences

The second method for creating correspondences is to allow CMap to create a relationship between any two features with the exact same name, irrespective of case. To do this, start cmap_admin.pl and choose "Make name-based correspondences." It is recommended that you define special evidence type to represent this automated correspondence. This method is likely to be both incomplete and overly-optimistic. For example, it will create correspondences between features called "Centromere," which is probably not desirable. The name comparisons happen on both the "feature_name" in the "cmap_feature" table as well as any aliases from the "cmap_feature_alias" table. Every feature has each of these fields compared to those fields for every other feature, so this can take a long time to complete. For all of its problems, this can be a good way to get started, and you can pare down the correspondences from there.

By default, CMap will only compare features of the same type when making name-based correspondences. You can expand the feature types considered by adding appropriate lines to "cmap.conf." This is documented in that file; look for the string "add_name_correspondence" and follow the directions there.

3 Import

The third way to create correspondences is to import them. Simply create a tab-delimited file that lists the names (or accession IDs) of two features and some evidence to support the correspondence. For more information on the format of this file, execute the following on your system:

perldoc Bio::GMOD::CMap::Admin::ImportCorrespondences

This method is the surest as you are always directly controlling what gets created.

LOADING THE CORRESPONDENCE MATRIX

The data underlying the correspondence matrix is all precomputed. As it is an intensive operation on seldom-changing data, it was determined to cache the pair-wise comparisons of all the maps in the database into a (very denormalized) table that would subsequently optimize the many calls for this data. Because of this, it is necessary that you remember to reload the matrix whenever you alter the number of correspondences in the database. To do this, execute cmap_admin.pl and choose the "Reload correspondence matrix" option. The only option is to completely truncate the table and reload it from scratch.

ESTABLISHING DATABASE CROSS-REFERENCES

See the "attributes-and-xrefs.pod" document.

VIEWING DATA

Now that we've stepped through the basics of setting up the CMap data, let's go over some of the more basic operations of curating the data.

VIEWING ALL MAP SETS

From the home page of the web admin tool, you'll see that you can "View Map Sets." Selecting this link takes you to the "Map Sets View" page where you'll notice three drop-down boxes that you can use to narrow the selection criteria for the map sets being displayed. You can restrict them by species, map type, and whether or not they are currently enabled. There is no automatic submission of the form when you make a choice (as you might want to use more than one criteria), so be sure to hit "Submit" when you've made your choices. As noted before, you can re-sort the data by clicking on the hyperlinked column headers. Every object in the web admin tool can be "viewed," "edited," and "deleted" by the respective links which are usually displayed to the right.

Note: On pages that may return a large record set, the data is "paged" and can be accessed by moving through the pages of the data. On all these pages, the total number of records is displayed along with which are currently being shown. This is the section that looks like "52 records found. Showing 1 to 25." (The page size is determined by the "max_child_elements" option in the CMap configuration file which will be discussed shortly.)

VIEWING A MAP SET

Choose to "view" one of your map sets. You'll be taken to a page which lists all the data in the map set table as well as a summary of all the maps associated with the map set. Notice that you can change any of the data for the map set by clicking the "edit" link at the top, or you can delete it from here by choosing the "delete" link.

Note: Deleting an object that has dependencies will cause the dependencies to be deleted as well. So deleting a map will delete all the features (which will delete all the correspondences which will delete all the evidences [but not evidence types] supporting the correspondence). Deleting a map set deletes all the maps (which deletes all the features which ... you get the picture). Basically, just be very sure that you want to delete something as it can have cascading effects, and, as noted earlier, THERE IS NO UNDO. There is, however, the option to dump your data before messing with it exists, giving you the ability to recover. This is discussed in the POD documentation of cmap_admin.pl.

VIEWING A MAP

Choose to "view" a map to see a summary of all the data in the map table as well as all the features on the map. Notice that you can restrict the features displayed by their feature types. You can also search for a feature; the search option on the map view page automatically restricts the search to the current map being displayed.

EDITING A MAP

If you click on "Edit" while viewing a map, you will be taken to a page with the following fields:

  • Map Set:

    Shows you the map set to which the map belongs

  • Acc ID (opt.):

    The map's accession ID.

  • Map Name:

    The map's name.

  • Display Order (opt.):

    The order in which to display the map in relation to the other maps in the set. If all the display orders are the same, then the maps will be sorted alphabetically by their names.

  • Start:

    The map's starting position.

  • Stop:

    The map's stop position.

VIEWING A FEATURE

Choose to "view" a feature from the map view page. You'll be presented with all the data stored in the feature table as well as the feature's aliases, correspondences to other features and the evidence types supporting the correspondences. You can also add a new correspondence by clicking "Add Correspondence" and following the directions discussed earlier in section 8.

EDITING A FEATURE

If you click on the "Edit" link from the feature view page, you'll be presented a form with the following fields:

  • Map:

    The map (described by "species-map set name-map name") on which the feature occurs; also a link to return to the overall view of that map.

  • Acc ID (opt.):

    The accession ID.

  • Feature Name:

    The feature's primary name.

  • Feature Type:

    The feature's type.

  • Is Landmark:

    Whether or not this feature is considered a "landmark." Landmarks can be used by the end user to show maps less dense with feature labels, tagging only those that you as the curator have decided are the most important.

  • Start:

    The start position of the feature. Must be less than the stop position.

  • Stop (opt.):

    The ending position of the feature, if any.

SEARCHING FOR A FEATURE

Often you will be interested in finding an individual feature in the database without having to navigate to the map set, then the map, then page through until you find the feature. From the home page of the web admin tool, you can choose the "Search for a Feature" link. The "Feature Search" form gives you four fields:

  • Name:

    The strings you wish to search for. Separate multiple search strings with spaces or commas.

  • Search:

    Search for the string(s) either in feature's "feature_name" (and aliases) or feature_acc.

  • Species (opt.):

    You can restrict by species.

  • Feature Type (opt.):

    You can restrict by feature type the features which will be searched.

  • Map Acc. ID (opt.):

    You can restrict the search to only those features falling on a particular map by specifying the maps accession ID.

CUSTOMIZING HTML TEMPLATES

All the HTML displayed by the application is contained in the templates. These templates are processed by Template Toolkit to produce the user interface. For the most part, the files contain straight-up HTML and can be altered to your heart's content. You could probably even pass off the care and feeding of these templates to a non-technical person (as this was the idea behind having no HTML in the code). The only functional parts of the templates lie in between the many "[% %]" tags, and these are often quite self-explanatory. If you can't figure out what to change on your own, then check out http://www.template-toolkit.com/ for the documentation (or type "perldoc Template" on your command line).

ADMINISTRATION USING cmap_admin.pl

The cmap_admin.pl script contains full documentation in POD format. To read, please execute "perldoc <script_name>".

That will describe how to use cmap_admin.pl using command line flags for scripting.

However, here is a description of the menu options that cmap_admin.pl provides.

1 Change current data source

Changes the data source currently being used. This choses which configuration file the script will use.

2 Create new map set

After a species has been created using the web admin tool, map sets can be created using cmap_admin.pl

3 Import data

Allows import of data from files. The files can be tab-delimited map data for an existing map set, tab-delimited correspondence data for existing features or the experimental CMap objects (which are created with the export function). To see more information about the tab-delimited format required, see the documentation that is in the modules by running "perldoc /path/to/Bio/GMOD/CMap/Admin/Import.pm" and "perldoc /path/to/Bio/GMOD/CMap/Admin/ImportCorrespondences.pm".

4 Export data

Allows export of data to a file. The data can be represented in a tab-delimited text file, as a series of sql insert statements or as the experimental CMap objects.

5 Delete data

Allows sweeping deletion of maps, map sets or correspondences.

6 Make name-based correspondences

Allows the creation of correspondences based on feature name similarities.

7 Reload correspondence matrix

Reloads the lookup table that the matrix uses. This is vital if the matrix is to have current data after a data change.

8 Purge the cache to view new data

Purge the query cache. The results of many queries are cached in an effort to reduce time querying the database for common queries. Purging the cache is important after the data has changed or after the configuration file has change. Otherwise the changes will not be consistently displayed.

There are four layers of the cache. When one layer is purged all of the layers after it are purged.

- Cache Level 1 Purge All

Purge all when a map set or species has been added or modified. A change to map sets or species has potential to impact all of the data.

- Cache Level 2 (purge map info on down)

Level 2 is purged when map information is changed.

- Cache Level 3 (purge feature info on down)

Level 3 is purged when feature information is changed.

- Cache Level 4 (purge correspondence info on down)

Level 4 is purged when correspondence information is changed.

* Cache Level 5 (purge whole image caching )

Level 5 is purged when any information changes because this level caches the whole CMap image when a map is first displayed.

9 Delete duplicate correspondences

If duplicate correspondences may have been added, this will remove them.

This option is where to import links that will show up in the "Saved Links" section of CMap. The import takes an xml file.

An example file is supplied, data/sample_saved_links.xml. You will need to change the accessions (*_acc) to reflect your database even if you used the test data (since the test data doesn't specify accessions).

Here are the main elements of the xml file.

  • cmap_views

    Container element that holds all of the other views (aka links).

    • cmap_view

      Each cmap_view element represents a single link. Multiple cmap_view elements can be contained in cmap_views.

      The cmap_view element contains many options. The only required option is a slot with a "number" of "0". All others options are useful but will still load.

      - group (optional)

      This is the group that the link will be stored under. This will override any group name given by the controlling program (cmap_admin.pl).

      - title (optional)

      This is the title of the link.

      - comment (optional)

      This is the comment for the link.

      - slot (Requires slot with number=0)

      Specify a slot element for each slot that you want to include. The slot has an attribute of "number". There must be a slot with a number of 0 for this to link to load. For information on the slots, see "Comparative Map Construction".

      Each slot has either one or more map elements or a map_set set element.

      • map (optional)

        Each map that you want in this slot needs a map object (unless you are specifying a map_set. A map element has multiple elements.

        • map_acc (Required)

          The accession value for the map.

        • map_start (optional)

          This specifies where on the map the view should start. If omitted, it will start at the beginning of the map.

        • map_stop (optional)

          This specifies where on the map the view should stop. If omitted, it will stop at the beginning of the map.

        • map_magnification (optional)

          This specifies the magnification level this map should have (in relation to the size it would normally be). If omitted it will default to 1.

      • map_set (optional)

        • map_set_acc (Required)

          The accession value for the map set.

        Fo

      - menu_options (optional)

      The menu_options holds objects that resemble the menu options (see "LINKING IN" for list of options). An example is listed below.

      <menu_options>
        <ft_DEFAULT>1</ft_DEFAULT>
        <image_size>large</image_size>
      </menu_options>

Note: If there are multiple CMap installations on the same machine, you may have to specify the --config_dir option to be sure that cmap_admin.pl is using the correct config directory.

REDUCING QUERY CACHE SIZE

cmap_reduce_cache_size.pl

cmap_reduce_cache_size.pl will help limit the growth of the query cache files.

This script cycles through each CMap data_source and (using the Cache::SizeAwareFileCache functionality) reduces the size of the query cache to the value given as 'max_query_cache_size' in the config file. It first removes any expired entries and then if it is still over the limit, it moves to last accessed entries. If 'max_query_cache_size' is not set, it will use the default value in Bio::GMOD::CMap::Constants. It is suggested that this script is run periodically as a cron job.

VALIDATE CONFIG FILES

cmap_validate_config.pl

This script will test a config file to determine if it is valid or not. It currenly only tests the individual data_source files and not the global.conf.

Run it on individual config files.

$ validate_cmap_config.pl config_file.conf

The last line of the output is most important.

The config file, config_file.conf is valid.
or
The config file, config_file.conf is INVALID.

You can read through the output to find out why it is invalid, if that is the case.

The script will also output options that are missing but not required to let you know what other options you can add. It will also tell you if you are using any deprecated options.

cmap_matrix_compare.pl

The cmap_matrix_compare.pl script contains full documentation in POD format. To read, please execute "perldoc <script_name>".

LINKING IN

Most likely, you'll want to link directly into the CMap viewer from some other part of your site.

To link to just one map, make it the "reference" map by using the accession IDs for the map itself and optionally,the map's parent "set". Here's an example showing just one map on Gramene, showing all the feature labels and highlighting the feature "RM9":

http://www.gramene.org/db/cmap/map_details?ref_map_set_aid=cu-dh-2001;ref_map_aids=cu-dh-2001-1;highlight="RM9";data_source=Gramene;label_features=all

Following should help you create your own CMap URL:

URI

/cgi-bin/cmap/viewer?

URL Arguments

Required Arguments:

  • ref_map_aids:

    The accession ID[s] of the reference map[s]. Multiple map aids can be separated by a ":".

    Information about the map such as start and stop can be stored in this argument. See "Map Accession Information Format" for more details.

Operational Arguments

  • data_source

    The "name" from the "<database>" section of the individual configuration file identifying the database and config file to use. If none is supplied, then the config file identified as "the default_db" in the global.conf file will be used.

  • ref_map_set_aid:

    The accession ID of the reference map set

  • ref_species_aid:

    The accession ID of the reference species

  • comparative_maps

    Describes the comparative maps. See "Comparative Map Construction" for a full discussion of how to construct this argument

    Information about the map such as start and stop can be stored in this argument. See "Map Accession Information Format" for more details.

Arguments for Speed and Clarity

  • aggregate [ default: set in config ]

    CMap has the ability to aggregate correspondences into one or two lines. This is faster and produces more easily decifered results on views with many correspondences.

    Values: 
       0:  don't aggregate 
       1:  one line 
       2:  two lines
  • collapse_features [ default: set in config ]

    A "1" or "0" ("Yes" or "No," respectively) indicating whether to collapse spanning features of the same type and with the same coordinates to overlap rather than pushing out

  • ft_DEFAULT (or the depricated feature_type_DEFAULT)

    This will define the how to display feature types that are not explicitly defined in ft_*, overriding the config setting. Setting the features to be ignored or display only when there is a correspondence can help speed the page drastically.

    Values:
       0 :  Completely ignore features of this type.  
       1 :  Display only features of this type with displayed correspondences
       2 :  Display all features of this type

    Note: Both the ft_* format and the feature_type_* format to define the same feature type, CMap will appear to randomly decide which one to use. So please be consistant and use ft_*.

  • omit_area_boxes [ default: 0 ]

    This determines which area boxes are rendered. The area boxes create clickable areas on the image. Rendering all the boxes gives the most functionality but can be slow if there are a lot of features. The option to omit feature area boxes can help speed things up while leaving the navigation buttons. There is also an option to ignore all of the area boxes, which is not recommended.

    Values:
       0 : render all of the area boxes
       1 : omit the feature area boxes 
       2 : omits all area boxes

Menu Arguments

  • ft_* (or the depricated feature_type_*) [ default: set in config ]

    These define how to display each feature type. The "*" is replaced by the feature type's accession (example: ft_marker=2). Setting the features to be ignored or display only when there is a correspondence can help speed the page drastically.

    Values:
       0 :  Completely ignore features of this type.  
       1 :  Display only features of this type with displayed correspondences
       2 :  Display all features of this type

    Note: Both the ft_* format and the feature_type_* format to define the same feature type, CMap will appear to randomly decide which one to use. So please be consistant and use ft_*.

  • et_* (or the depricated evidence_type_*) [ default: set in config ]

    These define how to handle each evidence type. The "*" is replaced by the evidence type's accession (example: et_blast=2).

    You can use this to ignore or display certain evidence types. You can also filter evidence based on the score. If you do this, there MUST be an ets_* value for this evidence type accession.

    If a correspondence has multiple evidences of different types and ANY are displayed, that correspondence will be displayed.

    Values:
       0 :  Completely ignore this evidence type
       1 :  Display all evidences of this type
       2 :  Display if score is less than or equal the est_* score
       3 :  Display if score is greater than or equal the est_* score

    Note: Both the et_* format and the evidence_type_* format to define the same evidence type, CMap will appear to randomly decide which one to use. So please be consistant and use et_*.

  • ets_* (evidence type score) [ Optional ]

    These define the score that the evidence of this type are measured against. The "*" is replaced by the evidence type's accession (example: ets_blast=97).

  • magnify_all [ default: 1 ]

    This holds the multiplier to determine the height of all the maps. For example, if magnify_all is 3 than all the maps are drawn 3 times taller. This can be handy if you want to see more detail.

    Any individual map magnification is then added on top of this.

  • label_features [ default: all ]

    Which features to label, legal values are "none," "all," or "landmarks"

  • scale_maps [ default: 1 ]

    Allow scaleable (set by scalable) maps to be drawn in scale

    Values: 
       0:  don't scale
       1:  scale
  • highlight [ Optional ]

    A space-less, comma-separated list of feature_names to highlight

  • image_size [ default: set in config ]

    How large to make the image, legal values are "small," "medium" or "large"

  • font_size [ default: set in config ]

    How large to make the font, legal values are "small," "medium" or "large"

  • image_type [ default: set in config ]

    The output type of the image, legal value are whatever image libraries you've installed (usually at least "png" or "jpeg" or "svg")

  • min_correspondences [ default: 0 ]

    The minimum number of correspondences to require between two maps

  • ref_map_start [ default: start of map ]

    The start position of the reference map. This will only be used if there is only one reference map.

  • ref_map_stop [ default: end of map ]

    The stop position of the reference map. This will only be used if there is only one reference map.

  • corrs_to_map [ default: 0 ]

    This determines where the correspondence lines are terminated.

    Values:
       0 : Lines connect features
       1 : Lines connect maps

Hide/Display Menus

These arguments allow you to hide or show individual menus. Set to 1 to display and 0 to hide. They are hidden by default.

  • refMenu: Reference Menu

  • compMenu: Comparison Menu

  • optionMenu: Options Menu

  • addOpMenu: Additional Options Menu

Just in case you see it

These will probably not be used in a constructed URL but they are discribed here in case you run across them and wondered what they do. Also, it's good to document them anyway.

  • flip

    This holds the accessions of maps that are flipped. The format is "slot_number=map_aid". Multiple maps are separated by a ":".

    Example: "1=map_A:-1=map_D:0=map_C:1=map_B"

  • comp_map_set_[left|right]

    This is the menu object that the form passes to tell CMap the map set of the comparative maps on the [left|right].

  • comparative_map_[left|right]

    This is the value that the form passes to dictate CMap comparative maps on the [left|right]. It holds the map accession of the map. Multiple maps can be specified by defining multiple comparative_map_[left|right] arguments.

    If the accession is defined as "-1" than all of the eligible maps in the map set are used.

    See "Map Accession Information Format" for details on the format.

  • sub

    The sub argument is part of a scheme to fix a JavaScript problem. The problem was that when a JavaScript, area-box button (on the image) was clicked, it would modify some menu objects and submit. Then if the user hit the back button, those menu objects would be the modified values, causing unexpected behavior.

    To fix this, modified_ref_map and modified_comp_map were added. When a JavaScript, area-box button is clicked, it modifies modified_ref_map or modified_comp_map instead of ref_map_aids or comparative_maps. That way, those fields always keep their original values.

    To determine if the "modified" values should be used, CMap looks at the value of sub. If there is NO value, then it assumes that it is a JavaScript link and uses the "modified" values preferentially over the regular values.

  • modified_ref_map

    See the description for "sub" above for the use of modified_ref_map.

    Information about the map such as start and stop can be stored in this argument. See "Map Accession Information Format" for more details.

  • modified_comp_map

    See the description for "sub" above for the use of modified_comp_map.

    Information about the map such as start and stop can be stored in this argument. See "Map Accession Information Format" for more details.

  • prev_ref_map_set_aid

    If the user is looking at a map (so they picked a ref_map_set_aid and a ref_map_aid), and then they choose a different reference map set, then current map is sent with the new (and different) map set. CMap uses this to compare the current map set against the previous. If they're different, CMap wipes out the map selection w/o making a db call to find out if the map is from the chosen map set.

  • prev_ref_species_aid

    See the above description for prev_ref_map_set_aid and extrapolate.

Map Details Arguments

Recently, the Map Details page has been folded into the regular viewer.

The URI of the Map Details page is "/cgi-bin/cmap/map_details?"

  • order_by [ Optional ]

    Order the details by this column.

    Values: feature_name, feature_type_aid, start_position

  • page_no [ default: 1 ]

    If there is enough data to be displayed on multiple pages, page_no dictates which page is diplayed.

  • comparative_map [ Optional ]

    Only display the comparative map data for this map or map set.

    Format:
       map_set_aid=set1
       map_aid=map1
  • action [ Optional ]

    Set the action to "download" to get a tab delimited file of the data.

Depricated Arguments

If you a are using these in a URL, please change the URL. They are not guaranteed to work.

  • ref_map_aid

  • ref_map_names

Map Accession Information Format

Drawing information about maps (cropping and magnification) can be inserted with the map accession in the following fields.

  • ref_map_aids

  • comparative_maps

  • comparative_map_right

  • comparative_map_left

  • modified_ref_map

  • modified_comp_map

The format is "accession[start*stopxmagnification]". Any position can be left out but the "*" is required. The "x" can be left out if there is no magnification.

  • start

    The start position of the map to display if the map is to be cropped. If left blank it will default to the start of the map.

  • stop

    The stop position of the map to display if the map is to be cropped. If left blank it will default to the stop of the map.

  • magnification

    This is the magnification value of the map. If increased to 2 the map will be 2 times taller, if decreased to .5 it will be half the size. All positive values are allowed but there is a minimum map size. The default is 1.

Examples:

cu-sl-1994-2[200*400x1]

Correct, crops from 200 to 400 with normal magnification

cu-sl-1994-2

Correct, no cropping with normal magnification

cu-sl-1994-2[*400]

Correct, crops from the start to 400 with normal magnification

cu-sl-1994-2[*x3]

Correct, no cropping with 3 times magnification

cu-sl-1994-2[200]

Incorrect, need the "*" otherwise we can't tell if you want this to be start of end.

cu-sl-1994-2[x3]

Incorrect, need the "*" because we say so.

Comparative Map Construction

To include some number of comparative maps, you provide them in the "comparative_maps" argument, which is a single structured string that lists all the comparative maps and their placement relative to the reference map. Here I'd like to introduce the concept of "slots," where the maps (or map sets) fall into a slot moving in positive and negative direction away from the reference map, which is in slot "0," like so:

-      -        -         -
|      |        |         |
|      |      - |  -      |
|      |      | -  |      |
|      |      |    |      |
|      |      |    -      |
-      -      -           -
          -1      0         1        2
  <----negative --+---------positive----->

The above drawing is representative of Gramene's genetic maps in slots -1, 0, and 2, and a physical map in slot 1. Slots are separated in the string by URI-escaped colons and the integral parts of the slot by URI-escaped equal signs. Like so

"comparative_maps="
+
    .-  <slot_number> + 
    |  "%3D" + 
  slot  |  <"map_acc" or "map_set_acc"> + 
    |  "%3D" +
    `-  <map_acc or map_set_acc>
+

"%3A"

+

   <next slot>

Here is a sample that puts "Rice-Cornell RFLP 2001-1" on the left (slot "-1") and "Rice-CTIR 2000-1" on the right (slot "1"):

comparative_maps=1%3Dmap_acc%3D423%3A-1%3Dmap_acc%3D1

The middle part of the slot is one of the literal strings "map_set_acc" or "map_acc." If you wanted to display just a single map in a slot, use the string "map_acc" and the map's accession ID. If you want a whole map set in a slot (e.g., a physical map set like "I-Map"), then use the string "map_set_acc" and the map set's accession ID. You can use the admin interface ("/cgi-bin/cmap/admin") to easily find the accession IDs.

USING CMAP MODULES IN CUSTOM SCRIPTS

There are several CMap modules that can be used by scripts outside of the cgi. To view more documentation on module use, execute 'perldoc Module_Name.pm".

Modules that are particularily useful:

  • Bio::GMOD::CMap::Admin

    Modules in the Bio::GMOD::CMap::Admin namespace deal with data manipulation. There are methods in Bio::GMOD::CMap::Admin to create/destroy all of the CMap objects (i.e. maps, features, etc). Bio::GMOD::CMap::Admin::Import contains a method to import tab delimited files. View the perldoc of each module in the Bio::GMOD::CMap::Admin namespace to find out more on how to use them.

  • Bio::GMOD::CMap::Drawer

    Use this module to create just the image and the image map. This would be useful if you wanted to integrate a CMap image on another page without the control stuctures. View the perldoc for instruction on how to use Drawer.pm.

  • Bio::GMOD::CMap::Data::Generic

    This module does all of CMaps database querying. If you need access to the database in code, there is probably already a method created for this. The advantage of using this module is that if the database changes, you won't have to change your code.

    To use you must create an instance of another CMap object. You can use Bio::GMOD::CMap or Bio::GMOD::CMap::Admin or just about any other module. Most of the Generic.pm methods require you to pass it this CMap object (as cmap_object) to give it access to the configuration options.

    To get access to the Generic.pm object, call "sql()" on the CMap object you created. Here is an example:

    my $admin = Bio::GMOD::CMap::Admin->new(
        data_source => $data_source,
    );
    my $sql_object = $admin->sql();
    my $features = $sql_object->get_features(
        cmap_object       => $admin,
        feature_type_accs => \@feature_type_accs,
        map_id            => $map->{'map_id'},
    );

SAMPLE DATA

The data/ directory has some example data. The sample-dump.sql.gz file contains a full set of CMap data. The tabtest* files also give some simple example files for importing.

If you would like to see other sample import data for CMap, you can find many examples from the Gramene site:

ftp://ftp.gramene.org/pub/gramene/CURRENT_RELEASE

MULTIPLE CMAP INSTALLATIONS

There may be occasion to run multiple CMap installations on the same server. To do so, use the customizable install options during the "perl Build.PL". The following is an example that creates a secondary install in the /usr/local/apache2/htdocs/cmap2/ directory.

perl Build.PL \
  PREFIX=/usr/local/apache2/ \
  CONF=/usr/local/apache2/conf/cmap.conf2 \
  WEB_DOCUMENT_ROOT=/usr/local/apache2/htdocs/ \
  CGIBIN=/usr/local/apache2/htdocs/cmap2/cgi-bin \
  HTDOCS=/usr/local/apache2/htdocs/cmap2/

Here is the breakdown of the options:

  • PREFIX

    This is the prefix for the web server install. The default is "/usr/local/apache/".

  • CONF

    This will create an alternate config directory to store the config files for this installation.

  • WEB_DOCUMENT_ROOT

    This is the absolute path to the document root of the web server.

  • CGIBIN

    This will create an alternate cgi-bin directory where the the "cmap" script. When using a non-standard cgi-bin directory, CGIBIN must be in WEB_DOCUMENT_ROOT.

    NOTE: It is important that the cgi script be executable by the web server. To do this in Apache 2, the following must be added to the httpd.conf.

    <Directory /usr/local/apache2/htdocs/cmap2/cgi-bin>
      Options +ExecCGI
      SetHandler cgi-script
    </Directory>
  • HTDOCS

    This is the absolute path to the directory where the cmap files will be stored. HTDOCS must be in WEB_DOCUMENT_ROOT.

The options not used in the example are "TEMPLATE", "CACHE" and "SESSIONS". These can also be set if so desired.

To access the CMap from the above example, the address will be "http://127.0.0.1/cmap2/". The cmap script will be at "http://127.0.0.1/cmap2/cgi-bin/cmap".

IMPORTANT NOTE: When creating the individual config files, the "name" field in "<database>" should be unique across the whole server. Using the same "name" for multiple config files, even if they are in different installs, can result in clashes over shared cache space.

OTHER NOTES:

  • The last install will become the default install for when no config_dir is specified. This should only affect cmap_admin.pl. To manually change the default install, modify the value of "CONFIG_DIR" in Bio::GMOD::CMap::Constants.

  • When using cmap_admin.pl, it will be important to supply a "--config_dir" value which points to the config directory to be used.

MISCELLANEOUS

In the "docs" directory, you will find two images that might help you understand the relationships in the CMap tables, "cmap-schema-graph.png" and "cmap-schema.png," both of which attempt to describe the tables and fields and their relationships. See also "cmap-schema-desc.html" for an HTML document describing the schema. These documents were automatically generated from schema definitions via scripts included with SQL::Translator, a set of modules which grew out of the author's constant need to make schema changes and quickly replicate them amongst the different test databases. The Oracle, PostgreSQL, SQLite and Sybase schemas were also generated by SQL::Translator from the MySQL schema, so if you see room for improvement, please relay them to the author, preferably via the SQL:::Translator mailing list. SQL::Translator is available on CPAN. For more information, see here:

http://sqlfairy.sourceforget.net/

TROUBLESHOOTING

Here are the solutions to a few common problems:

  • Error: File does not exist (image)

    This is a usually a permissions problem caused by the web server's /document_root/cmap/tmp directory is not executable by the web server. Change the permissions to allow the user that the server runs as the ability to execute and it should be fixed.

  • Error: .../cmap.conf: not a directory

    This is a also usually a permissions problem caused by the web server's /path/to/cmap.conf directory (or one of the parent directories) is not executable by the web server. Change the permissions to allow the user that the server runs as the ability to execute and it should be fixed.

  • CMap image does not respond to changes in config file or data displayed on the image is strange

    After changing a config file (or changing the data without using cmap_admin.pl), the query cache needs to be purged. Not doing so, often results in a mix of old and new information being displayed, which can be confusing.

    To purge the cache use cmap/bin/cmap_admin.pl. You can either use the menu system or by command line:

    $ cmap_admin.pl [-d data_source] --action purge_query_cache

    See "perldoc cmap_admin.pl" for more options.

  • Other

    Contact the CMap list, gmod-cmap@lists.sourceforge.net with a detailed description of the problem and the version of CMap that you are using.

    You can also leave a bug report for CMap at the SourceForge site for GMOD, http://sourceforge.net/projects/gmod/.

AUTHOR

Ken Y. Clark, kclark@cshl.edu Ben Faga, faga@cshl.edu

Copyright (c) 2002-6 Cold Spring Harbor Laboratory

POD ERRORS

Hey! The above document had some coding errors, which are explained below:

Around line 2381:

Expected text after =item, not a bullet