ICIS Workshop 2009
From ICISWiki
ICIS Developers Workshop 2009 Venue and Agenda
Venue: Singapore 1-3 14th floor, Bayer (Southeast Asia) Pte Ltd Office,
63 Chulia Street, OCBC Centre East, Singapore
| Date | Approx Time | Activity | PIC/Facilitator Reporter |
| Monday, 16 Feb 2009 | 9:00-9:15 | Registration | May/Sharon |
| 9:15-9:45 | Welcome Address | Ed Roumen | |
| 9:45-10:00 | Orientation (Facilities) Group Photo | May | |
| 10:00- 10:30 | Overview and Workshop objectives, Finalize agenda of workshop | Thomas Metz | |
| 10:30- 10:45 | Coffee break | ||
| 10:45 - 11:35 |
Activity Reports of ICIS Partners (10 minutes each)
| Thomas
(Reporter:Clarissa) | |
| 11:35- 12:25 |
New members key Interests in ICIS (10 minutes each)
| Thomas
(R:Arllet) | |
| 12:30-1:30 | Lunch | ||
| 1:30-3:15 |
Updates on different ICIS modules
(Some participants will be on another meeting) | Clarissa
(R:Mylah) | |
| 3:15- 3:30 | Coffee break | ||
| 3:30 – 5:30 |
Development of Other Modules
| Beth
(R:Bibiana) | |
| Tuesday 17 Feb 2009 | 9:00-10:30
| Database/Schema
| Sebastien
(R:Victor) |
| 10:30 - 10:45 | Coffee break | ||
| 10:45-12:30 |
ICIS Web
| Mylah
(R:Warren)
| |
| 12:30-1:30 | Lunch | ||
| 1:30 – 3:00 | Molecular Breeding Platform being proposed by GCP
Discussion | Graham
(R:Cameron) | |
| 3:00 - 3:15 | Coffee break | ||
| 3:15-5:30 |
Future Development
| Ching
(R:Connie) | |
| Wednesday, 18 Feb 2009 | 9:00-10:30 |
Quality assurance and Documentation
| May
(R:Fran) |
| 10:30 - 10:45 | Coffee break | ||
| 10:45-1:00 |
ICIS implementations and conversion
| Shawn
(R:Sandra) | |
| 1:00-2:00 | Lunch | ||
| 2:00 – 3:00 |
Inventory Systems
| William
(R:Grace) | |
| 3:00 - 3:15 | Coffee break | ||
| 3:15 – 5:30 |
Inventory Systems (Con't)
| William
(R:Grace) | |
| Thursday, 19 Feb 2009 | 9:00-10:30 |
Break-Out sessions
Other proposed topics:
| Casper |
| 10:30-12:30 | Discussion of the Break-out Topics | Casper
(R:May) | |
| 12:30-1:30 | Lunch | ||
| 1:30-5:30 | How to make ICIS more user-friendly | Ed
(R:Candy) | |
| 6:00 | Workshop Dinner | ||
| Friday, 19 Feb 2009 | 9:00 - | Future planning | |
DISCUSSION
DAY 1
Activity Reports of ICIS Partners
Development in IRRI (Thomas)
- Currently at the recruitment stage
- Created a stronger link between partners with the movement of Arllet(CIMMYT) and others(IRRI)
CIMMYT (Arllet)
- IWIS2 in Win SQL server (past)
- IWIS3 Central plus Markers in Access server (current situation)
- IWIS3 Public
- Establishing SMTA is a problem
- still using IWIS2 to curate incoming and outgoing seeds
- no under development classification
- IWIS3 database
University of Queensland (Sandra)
- Correction of records and inform CIMMYT
- CAGE
- ended in July 2008 & is renewed for the next 5 years
SPARC (Shawn)
- Breeders want summary info of their datasets
- Biotech Lab
- make them use ICIS
- created workbook templates
- interest is growing for ICIS
- Dr. Curtis Pozniak has problems running ICIS in his computer
Nunhems (Casper)
- Further development of Oracle-10 & VPD for ICIS v5.5
- Mini ICIS-programmers Workshop Nunhems-NL, April 2008
- Development of DST (Data Selection Tool) as RTV for Oracle+VPD
- Migration of first teams into ICIS v5.5 (internal called ICIS Global)
- Further development of Nunhems Breeding Help
- Development of daily check routine of database integrity
- Joined development with BCS on ICIS pre-login for Oracle+VPD
Bayer Rice (May)
- Full migration to VPD
- Had specified key users for ICIS (but not yet the entire target users)
- Created teams and groups for different countries and functions
- Key users are now loading their studies
New members key Interests in ICIS
Bayer BioScience (Sophie)
- Crop (Canola, Cotton, Rice and Vegetables)
- ICIS is already implemented for rice
- Goal is to develop molecular breeding platform across all crops
- - Analysis was done about the different available tools
- Tools used
- Canola- Agrobase
- Cotton - Prism
- Rice, Vegetables - ICIS
- Molecular Breeding Platform
- Breeding DB - phenotypic data
- Marker DB - Genptypic data
- On top of that, is the GBrowser for annotations and CMAP for genetic/physical maps)
- Mining layer (analysis/statitics/support for decision making)
- Requirements
- based on ICIS
- multi-user, ulti-crop, multi-site
- user-frienly, integrated, flexible
- secured data - read or write access to the data
- meeting the needs of all the crops
- integration of data mining tools
- Tools used
Canola Breeders, Australia (Cam Beeck)
- Funded by university, federal government and German agency
- started on 2001
- CBWA and ICIS
- relealized that data management requires full-time staff to handle it
- GMS - a paper was written about the genetic diversity in Australian canola and the ICIS COP confirmed the diversity
- Austrlian canola has basically 11 parents
- some issue about DH lines treated as F2's ( .ICP LST <LVL> command in Browse)
- it would be nice if this command is available from SetGen
- The goals is to use as much Genotype information when assessing GXE
- Goals :
- develope the DMS
- inventory management
- hire staff for ICIS Management
CIMMYT- Wheat Genetic Resources Center (Bibiana)
- Visits were done in IRRI to see the Genebank's operation
- Types of storage
- Active Collection
- Base Collection
- Activities
- Design of the label
- Adoption of InTrack with the help of CRIL Staff (Beth)
- Load data about amount of the accessions and many temporary staffs were hired to assist
- Worked with Candy for the modification of SetGen
- Next Plan
- Loading of the Passport information
- Viability of the active and base collection
- Graham : How many packets of seeds from the Genebank have you done the inventory already?
- Bibianan : About 200,000 packets of seeds
ICARDA (Zaid)
- since 10 years ago, we tried to adopt ICIS
- IFBIS is the well implemented for ICIS: 3 modules of ICIS were covered (GMS, DMS, GEMS)
- We are looking forward to collaborate with CIMMYT about the wheat data
- It is still a long way to implement ICIS but this year, we intend to move forward. With the wheat, we hope to cooperate with CIMMYT and use IWIS3 as central. To initiate its use for durum an dbread wheat
- improve awareness and use of ICIS in ICARDA
- all ICARDA breeders are now eager to use ICIS with special emphasis for wheat data
- Apart from ICIS, other systems are planned to link with it.
- LIMS
- Genetic Resources Information System
- Graham: Where is the wheat data?
- Zaid : Either in Excel or Agrobase
- Graham: Is that split between bread wheat and durum
- Zaid Durum is in Excel and some bread wheat in Agrobase
WARDA (Tchamba)
- crop is only rice
- key interests are in seed inventories, germplasm location
- integration of ICIS with Genetic Resources data and evaluation data
- evaluation data came from India and Africa
- a system is already developed in India
- current databases are in SQL server/Ms Access/mySQL
Updates of ICIS Modules
SetGen (Clarissa Pimentel)
Changes made in 2008 (Download presentation)
Documented on cropwiki.irri.org website:
- Next_sequence_of_a_name_type
- MAN
- Customized Data
- Attribute Inheritance
- Attributes as Documents/Files
- Import function forms
- Barcode Tool
- Add A New Germplasm
- InTrack Reserve Form
Comment by Shawn:...
Add a new germplasm - question to users:
- when you use this form, what would you like the default data to be
- note to candy: it should be blank
- this feature is not a batch feature
Add a new germplasm - question from users:
- for example there is thousands of accessions, how to add?
- use the import interface in setgen
- the format is documented (contact Candy); see Import function forms
- there is a a tool developed by William
- both tools should support the same format
Creating accession number using a prefix
- support for 5 character prefix
- why use any key at all- ask the user to input the prefix
- the problem is genebank in CIMMYT uses 5 different prefixes
- form check: limit the user to choose only from those 5
- if there is only 1, make it the default
- this is related with the installation, put in INI file
Create barcode
- what will happen when there is 1 barcode for each unit
- choose from drop-down list to choose
- Barcode Layout
- developer(candy) need to incorporate, for now only 1 layout
- used layout by Bayer
GMS Search (Beth Sison)
(Download presentation) Documented on Changes made on GMS Search Tool
ICIS Workbook (Warren Constantino)
Data Validation Tool (Maria Corina Habito)
Q&A:
Thomas: how do you consider security issues..those are very powerful tools .what kind of authentication mechanism do you user for checking?
cing: i use the windows authentication..
arllet: it doesnt use the icis..because it has an access in the database bec can set access to a certain database
ching: i use the database to check
arllet:you use the Users table?
martin : why do we used this tool..
graham..: bec of the version
ching: bec of the application usually
graham: mostly on DSN stuff, if you connected to the database correctly
graham: have you tried it on a magic computer that you couldnt do anything..one computer where nothing of icis would run [chuckles]
arllet: it just doesnt like icis to run
martin: the sedcond q is why do you need the ini file fcor the data validation tool?
casper: bec in the ini file which databaseto connect to..why use this tool? well, we use this to check our Oracle databasde to check the updated database
ruaraidh: how is the procedure to corect these data, and is there a recording/tracking how and why is this data been changed/corrected?
martin: this is a good Q..icis is the tool to correcting or curating the the data.
ching: at the moment in the GMS,in the changes table
graham: if it is GMS, you send them to WIlliam right?how will you know which center requests this changes?
ching: i included the icis user in the results
sandra: <explains on how the the requests are given and sent to cropforge>
ching: it's usually for the applications.if you use setgen and browse to correct the data, it gets recorded in the changes table.right now we dont use that in the DMS..
Installation Diagnostic Tool (Maria Corina Habito)
Q&A:
tom : in your tool, you treport on errors.do you also report on successful checks?
ching : it is indicated in the tabs
ruaraidh : does it check data?
ching: only in the schema..actually in the data validation tool, it checks for the data
ICIS Administrator's Tool (Maria Corina Habito)
Q& A:
Shawn: give more description of what the INI keys mean
CHing: yes, on the other tab, it will show the description of the INI key
graham: will it have a nicis installer, will it for exmaple, i want a wheat breeding program to get an empty database from the central?
Data Selection Tool - DST (Casper)
Q&A:
Graham: If a list has 1500 germplasm entries, goes it then to the next plate?
Casper: yes, it goes to the next plate, each plate a new sheet in Excel
Graham: thank you again casper for showing us the way..
Data Compression Tool (Shawn)
Q&A:
Ruaraidh : why IRIS is a challenge?
Shawn: i dont know.seems easy for IWIS, but looks difficult on IRIS.
Development of Other Modules
Genetic Resources Information Management System (GRIMS-IRRI) (Beth)
Q&A:
Graham: Are you sharing the local data into the Central?
Beth: Yes, It is uploading the Central DB & Local DB
GRaham: when?
They said once a month
Then Seed Acquisition with GMS Search
Modules used SetGen/Workbooks/Intrack/GMS Search
ZID: Is this system based in the old version of management system at IRRI?
Beth : Yes, it is based in the existing system
Zaid: there are several systems now
Hamilton: Genebank has beed used this management system long time ago,
Graham: Sorry, again about Issues?
Hamilton: Something about characterization
Graham: Move the characterization into DMS DB.Use ICIS.that's a challenge to you, Beth.
Discussion on People, Institute and Location (Ruaraidh)
Q&A
Zaid: how much of this is related with ICIS modules?
Hamilton: <discussed on how it is>
CIMMYT - Wheat (Bibiana)
Presentation of WARDA Genebank Overview(TChamba)
Q&A
Arllet: i dont have a Q but a comment..The development tool that you use, Beth also uses that for the Inventory Tracker, including Rave Reports for barcode generation.
Graham: THat germplasm has IT numbers?
Tchamba: Yes
Graham: global register system
Tchamba: They dont have manegement application, for stock management.
Graham: Problem with the label, because the key is the acession number
Tch: They keep the accession number for stock and regeneration
Graham: Do you keep the old database?
tch: yes
G: We may have a problem with that because we have a concept of a GID...[further explains on how GIDs/germplasms are related]
Tch: there are many missing data, they are trying to update
DAY 2:
Database/Schema
Changes in the Database Structure for ICIS 5.5.1 (Arllet)
- brief discussion on previous structure
- no table to represent
- added tables levels obsunit represtn
- level - contains the levelno (p key)
- factorid
- represtn - represntation in the effectid
- represno, effectid, represname
- obsunit - observation unit of the effects
- ounitid studyid effectname
- normalize tables
- prblem to multi access issue, two users might get the same representid
- check if id is existing
- added ulrecid in INSTLN
- listdesc length inc to 255 in listnms
- source length inc to 255 in listdata
Q&A
SHawn : Don't we have new fields in the STUDY table for levels?
Arllet : Yes, but we have discussed that on last year's WShop
Shawn : oh ok, I wasn't there..
ICIS Database Access - Linking of ICIS DB to the GCP Domain Model (Richard)
questions:
Q: what will happen after?
A: step back and sort out things
Q: allocation of space based on the data model??
another Q: doing too much in the model?
Comment: nothing has been implemented;
Answer to Comment : model has been in active use, plugging of diff dbase into the model
ANother Comment :new breeding platform bringing all molecular breeding projects together
Multi-user Access and restriction - General Solution (Shawn/Ching)
questions:
Q1: level security for breeders? administrators?
explanations...
Q2: security issues?
explanations...
Q3:what are the user requirements?
explanations ...
Multiple crop in a single database using Oracle (Sebastien)
- goals
- gather all icis dbase insto a unique repository
- apply security based onf grp osisers and crops
- keep the isics dvelpoment
- sec solns
- apply the sec at the application level
- use a view on the top of the table
- mult of roles and privileges
- sec is applied at the tale level
- vpd
- vpd is defined
- 3 components policy, policy func, predicate
- implemented in the orcle engine
- ex: predicate, icis setgen run query, vpd modifies the query by adding the predcate
- security requirements
- user is described by description level
- each level allows access to as set of data
- date are private to user
- can be shared to a usergroup aor public
- security tbles -> vpd level, vpd level values, vpd users, icis session
- sec fields -> xtra fields , insid, insdate, updid, upddate, vpd_public
- filled by oracle triggers automatically
- sec funcs - > same fields are available in all tables, generic functions, c
function can be called by multiple policies
- return a specific predicate based on the access rights
- sec policies- determine when and how to apply policy
- specific to a schema proj
- callss a vpd function
- contrl one or more statement
- examples of ICIS VPD
- icis constraints
- instln table - dbases are distinguished by diff ids
- only one row in ICISL.instln must be present
- soln added new field val_id
- users table - does not allow duplicate names
- soln apply vpd filetering on the users tables
- for the current connected person, only retrieve one icis user id
- conclusion
- gather all icis dbases into a unique repository
- apply security rules based on grps of users and crops
- keep the icis developments as generic as possible
- security mechanism on dbase level
- in production for vegetable and rice
questions:
- one user with mult user level?
Answer : explanations...
- apply vpd in whole system?
Answer : explanations...
Multiple crop in a single database using Postgres and MySQL(Ching)
- backgrnd
- mysql 5 - grant command,rowlevel sec using views
- pstgres - veil sec addon
-provides an api allowing contrl of access data at the row and col level
-security is always present
- veil security model is assigned
- implement and test the oracle vpd strategy
Q&A
Q1 : is it implemented?
A : just initial research
Q2 : efficiency in view creation?
A : problem if view is not updatable, in adding records?
Q3 : different security levels from diff organization. e.g IRRI security?
A : login id valid in all servers of different dbases?
ICIS Web
New ICIS web portal (Martin/Mylah)
Documentation on Koios on GCP Pantheon website
Available Applications
- See Pantheon Components Inventory: http://pantheon.generationcp.org/inventory/gen/
Development release (includes partially integrated pedigree viewer):
- IRIS
- IWIS
- IMIS
Pedigree Viewer demo by Martin (See Belisama Website)
Study Browser(Mylah)
- See Cropfinder included in Pantheon Components Inventory: http://pantheon.generationcp.org/inventory/gen/
List management, germplasm and trait list management (Mylah)
- See ICIS Websites included in Pantheon Components Inventory: http://pantheon.generationcp.org/inventory/gen/
Integration of CropFinder into Koios (Mylah)
- See Cropfinder included in Pantheon Components Inventory: http://pantheon.generationcp.org/inventory/gen/
Display of Georeference(Martin)
- See Belisama Website
Q&A
Q1 : Support for IE other than ForFox?
A : Yes.
Q2 : How hard is it to setup as a stand alone?
A : Not yet done but will provide a package that would install as a stand alone.
Q3 : Data entry already?
A : No, not yet started. Plan for this year is for pedigree curation.
Feature Request: Version #, What's New, and Bug Report in the web page
Molecular Breeding Platform being proposed by GCP
Q & A
Ruaraidh: what is going to be the procedure to properly engage use cases? are you going to involve users to create these use cases?
G: each of these use case projects have strirring up committee..i divided my team to inter-act with projects that they work more closely. <more discussion on how to do this>..we are open to ideas on how to do this. I would want to have team mates be on the breeding program where they really are working in/familiar with.
[more explanations..]
Ruaraidh: so thomas is expected to talk to the breeders?
G: yes, precisely, he has started with doing that already..if this project will work with the ICIS commiunity, then it would be nice
Next Gen ICIS (Martin/Graham)
Q & A :
G: Q's Fran on how shew can express her concerns on using a web-based system
F : Explains on how she initially felt on having a web-based system, not having a stand-alone.But after Richard explains that she can have a standalone system with a web-based applicaiton
G: but you can do it..but you need to make up whether to use a standalone or web based
G: in my point of view, we can have a web-based on a local computer.. [further explains about how to do it]
Martin : explains on how using a web-based application can be used in a local computer...
G: I asked casper how the breeders
C: they dont have a copy of all data but just a subset of the data, those that what the breeder needs
F: on their situation, the nbreeders go to other countries and doesnt bring all the data, but just hte data that they need..
M:the option that they can use excel..anykind of applicaiton can be a
G: to sophie Q on web platform
Sophie : explains on they need access to data,some breeders need only a subset of the data that they need. We provide what the user needs - > cross platform --> excel if they need that, and web-based
Thomas: explains difference between web-based and stand-alone..
Zaid: Q's on what is the bandwidth that we are talking about..
M: Answers that we talk about DB connectivity, it doesnt matter which applicaiton we have, SA or web-based..further explains...
Thomas : explains the disadvanteage of stand-alone applicaitons
G : well, in web-base, you have to take care of which browser to user
Aaron: explains about coonectivity, not just icis, but also breeding tools...
G: so what you are saying is that we are focusing on catching up not connectivity...
Aaron : yes
THomas : discusses on trends...
Aaron : explains on how the computing giants would eventually influence us...
Thomas : explains on how "political decisions" are being done, so there are reallfy
Sophie : as for them, their choice of "techhnology" is based on data that they need
casper: explains on how they need data on the net and on the local machine
Shawn : explains how their breeders are screaming on the idea that their data are analyzed on the internet by others...asks how the idea of having data on the net would be handled..explains how data will not be updated from intranet to internet
G: explains how running on the internet, there will be an update problem..in terms on the durability of the software, it will still be running as long as the server is running,..
G: my Q is that if you have a web
Thomas :can i propose to slice the gorilla in to two pieces bec there are two sides of it..
Aaron : agrees on shanw's sentiments but thinks its more on "cultural" practices
Fran : explains on what their concern is on flexibility...
Ed : explains that it is not on cul
Future Development
Customization of the Graphical User Interface (Thomas)
Q & A
Warren: is there a design like that for the handheld?
Thomas: depends on the language you want to use...provides graphical interface integration...
Graham: Could we do it in a function level and secondly in the workflow, third users....
Thomas: basically combination of all 4, the role should be reflected in GUI...
Martin: This year's focus is to meet end-users customization
Mylah:Already have configuration files that state the task needs and collection of the modules
Martin: Separation is possible for application but not the priority
GRaham:Genebank has nice functional category
R:...
Martin:Which interface are we talking about stand alone or web interface (new ICIS web gen.)
ED:Next generation
R: user-friendly
Ed:Naming convention?
Data validation rule could be customized?
Ching: Dr. Halmitlon is mentioning on capturing then data error. Then put it in data installation
Integration of statistical analysis (Thomas)
Q & A
Martin:Uses of R application, data using R?
Thomas: designed for advanced user
Graham:users will select the data and get the result back
Ruaraidh:Where this concept comes from, for integration of analysis into data mgt?
Arllet:To provoide dtamgt tool to specify export data to their analysis prog.
Zaid: many are using gemstat or SAS, should we constraint ourseleves to R?
Graham: No
Thomas:Reason for using R is this can be distributed freely
DAY 3
Quality assurance and Documentation
Validation of coding and documentation standards using JUnit(Martin)
Validation of coding and documentation standards using DUnit(Clarissa)
Testing protocols: Use of Test Cases (May)
User documentation and Help, wiki/multimedia (Thomas)
ICIS implementations and conversion
GPG2 Cross referencing of Rice and Wheat Germplasm
Speaker:Tom Hazekamp
Project aimed at upgrading the CGIAR centre genebanks and the standards of managements of the collections.
Objective of crop registries is to consolidate a list of accessions and associated information for a virtual global collection of a crop and establish links with GPG2 projects.
There are about 200,000 rice accessions in the first phase.
There are about 190,000 wheat accessions in the first phase of the wheat crop registry, coming from CIMMYT, ICARDA and USDA.
To establish the duplication between the registries, the passport data needed to be standardised, then selected a set of combinations for analysis to produce a similarity matrix.
Tools development for the task: Cross Referencing Tool
Results: in rice, out of 200,000 accessions, almost 40,000 had one or more replicated accession in other collections
Next step is to integrate the data into IRIS. A visualisation tool for data entry and data edit of pedigrees would be very useful. Errors in IRIS data are making data integration difficult. Data quality control is needed and the workflow of data entry needs to be reviewed.
Tchamba – how do you cross reference location data?
Tom – make a score between 0 and 1 – what is the difference between longitude and what is the diff between latitude – depending on what the difference is, assigned a score between 0 and 1. If they are missing, they are ignored and not included in the calculation. You might have other information such as donor number which you can follow to get more information.
Uploading pedigree updates into IWIS3 and IBIS
Speaker: Shawn Yates & Sandra Micallef
IWIS3 is the ICIS version of the IWIS2 system at CIMMYT. Past issues in the conversion : inconsistency between versions and inconsistent time intervals between versions.
Some problems that we are encountering at the moment: conflicts in pedigree information should be corrected by the local administrators not the central administrator as they are the ones who have contact with the local ‘expertise’ on the pedigrees. Also settings in value of GPID2 affect the calculation of COPs, so we need to be consistent.
Another problem was that not all tables are checked for upload from local to central – some data is missed and causes problems later on. An administrator tool would help in this process.
Regional Central databases: intermediate database between Central and local databases – if it is going to be adopted by more people, we need to document it.
Martin Senger: what is the difference between local and central? We need to define the rules of how to share data between locals (share of GIDs)
Graham : the unique identifier for a crop is the crop installation, the user id and the local gid number of the line. This identifier (other than the GID) is preserved, no matter where the data is transferred.
Browse tools need to be integrated in SETGEN or produce a more user friendly interface. We need more user documentation on the WIKI site with examples. We also need to establish protocols for entering germplasm and upload procedures for new users.
Ruaraidh: what happens if you think you know the source of a line, do you still put GPID2 as 0.
Shawn : yes, because it effects the COP, then add an attribute with a note
Graham: you can include a similarity score as an attribute
Deployment of Multicrop ICIS mirror sites
Speaker: Marten Senger
How to package together the application? 3 packages are in consideration.
1. how to package the database?
2. Deployment – falls under cropforge, which will have a user interface to connect to the database, with the ability to change databases
3. tomcat server included – ability to select whether you want to install it on the PC
This is the vision of the package.
Thomas: is this platform independent – for linux and windows?
Marten: yes
Shawn: if you have multiple locals for wheat, how is this system going to work?
Graham: the original idea of web network was to move away from the central/local notion and move to a network of databases. First you identify which data bases are plugged in. The local identifier allows this to happen as it is not only a one dimension increment of GIDs. There are still decisions to be made about this in the MBP
Validation of IRIS data
Speaker: Ruaraidh Sackville Hamilton
Example from IRIS database of errors and corrections suggested
Too many errors – more than 1M in GMS table
How to proceed to speed up data correction rate?
Need to set up the right environment – find the errors, what the best way to correct it is and how to do it. Data validation rules have been set for data entry.
What does a GID represent: A cross, or a collected sample or an accession? All of these and more, but rules aren’t usually followed.
Can a name be a germplasm ID? – if the NSTAT =8, NVAL uniquely identifies GID at GLOCN.
Arllet: most of the GIDs in IRIS were created because the users could not figure out which line they were dealing with, so they created a new one instead of selecting an already existing one in the database.
Shawn: are the changes being recorded?
Ruaridh : yes
Speaker: ARllet Portugal
Migration of legacy IWIS to ICIS
The IWIS Wheat pedigree management is good for searching and modifying pedigrees. The Wheat Data management system (similar to DMS) produces statistics.
The conversion program was produced to migrate data from IWIS to IWIS3. The Genebank is using ICIS to manage seeds.
Some issues with data migration: pedigree information is not the same as the one that comes out of IWIS
Graham: this arises because CIMMYT name their crosses, example BABAX is a cross name as well as a line. The IWIS pedigree expansion is such that it stops when it meets a named cross.
Arllet: This will be overcome by introducing another name type. The DLL needs to be changed too
Ed: may be a good idea to introduce the idea of working with seed lot numbers rather than names and crosses,
Parsing of selection history is very good in IWIS and should be adopted in ICIS
WINS – system to manage shipping – not available in ICIS
Dispatch module and Genetic resources – can be applied to CIMMYT
Planning to release IWIS3 once this year
Towards full adoption of ICIS at IRRI
Speaker: William Eusebio
After discussions with other database managers, William wanted to find why the breeders at IRRI were not using ICIS to manage their data. The goals for the next 2/3 years are to train the scientists to be able to use querying tools to retrieve information from database to make decisions for their experiments, and to have accurate information about germplasm movement between the IRRI sections.
It is important to train the IRRI staff to used ICIS and for ICIS developers to deliver an improved and user friendly software.
INGER data in ICIS – nursery lists from 1973 to present and characterization data from 1975 to present. Also in the IMS is fully implemented for seed stocks monitoring and withdrawing of seed requests.
Yield trial data from the plant breeding group and laboratory genotypic data are still not in ICIS. The PBGB Data management team is conducting training to breeders and researchers to use ICIS.
Ed: Do you give training to scholars who are visiting IRRI? So when they go back to their institutions they take the knowledge of ICIS with them?
William: Anyone at IRRI who are interested in using ICIS can be trained
Inventory Systems
CIMMYT Wheat Genebank, Inventory Experience (Bibiana)
Ed: What do you consider the lot id?
B:
Ed: What does the barcode represent?
B: The lot id.
E: If I request for an accession name what you’ll provide me?
B: It will give you all the information related to the requested acc.
B: Maintaining only the stocks in the local db
B:What happen if I need to put the batch with 100 accessions?
Bibiana doing GMS search…
Graham: Can we see stocks there?
A:The stocks info can be viewed also in the GMS Search module.
Ruraih:What you do with the viability data
B:Nothing, just generating the data, but would like to see the info
R:Wheat are not putting viability data in the DMS,just for management…
G: In the latest template , the testing date has already moved. I think you’re not using the latest version of workbook.
B: It is the latest
Failure of ICIS is in the Implementation
Involved Parts of the process
1. Decision makers
2. Users
3. Programmers
What strategy?
M: the developers are also human
Ed: If the 21st century will require 12 mouse clicks, I’d rather use paper that only require 2 lines. The functionalities are there but not in the proper order.
NRSH: bring out the Global team to discuss. Coordinate with GRIN
The lot id is in the seed packets. The seeds are stored in different packets but have the same GID for the same crop year.
SeedMan (Zaid)
In ICARDA there is a dedicated seed unit. They are using Seedman. This handles seed production and multiplication. Does not include Genebank inventory.
Seed unit can produce reports on production, seedlot data, field data, insecticide/fungicide applcn.
Showed report on production (please refer to the ppt presentation)
Storage Mgt
- Produce reports in
- Distribution
- Balances
- Distribution
Seed labels
Lot histories
Lots entered
Costs
G: Is it all crops in one?
Z: Different system for diff. program
Nunhems Breeders Seed Inventory(Casper)
Casper: The Nunhems inventory does not use InTrack because it does not fit with our workflow
G: Create a warehouse for the viability
NRSH: We already have. Why load it to dms using workbook when it is not needed to be known to public?
Inventory stocks can be seen in GMS search and in setgen The system is straightforward Useful for the breeder in our case
GRIMS-IRRI (Grace/Beth)
Seed Management
Ed: What do you measure? The amount you will take?
NRSH: No. It is the actual amount left.
Ed: What’s the Sv_id?
NRSH: ?
Arllet: In Intrack the amount taken is the one measured. The amount left is more important than the amount taken.
NRSH: Yes. We don’t want to incur negative amount which will result if the amount taken is measured.
Further discussion (CIMMYT,IRRI,BAYER)
Intrack: CIMMYT would like to adapt the GRIMS interface.
Discussion Arllet:How do you create a request from the GRIMS? Beth:It goes to the seed request table, Can get it from the list/file The GRIMS uses ORACLE as backend CIMMYT has no ORACLE installed
Arllet:How do you find GRIMS IMS interface, will it suit you? Request received by CIMMYT is in different kinds (cross,name,gid…)
Replace the “get from file” to get from the list
How to generalize the GRIMS from the other crop? Preferred ID/Abbrev name
How will you know if there is no seed stock for 1 entry in the list. In the stotype status: U – means unavailable.
“Select IRGC request number” can be the list name
Step3 tab: Accno, cropyear and storage correspond to one LotID
There is a plan to have a central seed stock system for the two groups: Inger and GRC
Institute table of GRIMS has duplicate entry, redundant. They are not yet using ICIS’Inst table. Arllet’s recommendation: yes they can be edited.
Institute type? – stored in the UDFLDS, you can edit them. Talk to INGER to merge the similarities.
LOCID – Arllet: yes logical to put this INSLEVEL -
DAY 4
Introduction to GEMS (Arllet)
Refer to: Gene Management System 5.4 GEMS Schema v5.5, GEMS Catalog, GEMS Schema v6
Graham : explains further on GEMS, specifying that it has no Attributs table
Q&A
Graham : asks if this application uses MSAccess
Arllet : yes
Graham: (to Arllet) Does the GEMS interface allows data entry?
Arllet : yes..further explains that there might be a slight problem.
Graham : this is nicely done in the wiki (comment on procedures to using GEMS)
Arllet : problem is it doesn't link back to tables
Graham : this is the data model which has a new model
Aaron : does this data model show one locus to one molecular variant?
A: No.
Graham : explains on the problem with implementing GEMS is getting an exisiting data.
Shawn : but this really works well for us.
Grhaam : [to Aaron] what do you use?
Aaron : we use Gramene.
Graham : you mean that you get the schema and use it?
Aaron : yes
Ed: asks casper if they user the same GEMS schema?
Casper : No
Ed : linkage group is mapping within the group.Linkage map are not absolute
Graham : map can be the for the whole linkage map
Graham : to assemble one genome you need at least 11 or 12 maps
Ed: For the breeders, the linkage map is important.
graham : [to martin] how can you generate a schema out of this model?
Martin : explains on how a schema can be done through brainstorming...make a plan to make it useful.
Graham : explains that he thinks that this model is ver 6...Shawn and jemi is using it. What drives a user to move from the previous version to this new one is if the user is having problems with the current one.
Martin : the schema is not for breeders.
Shawn: we can't see what germplasm it is run.
Graham : explains why GEMS is there because the gene is a linkage to a trait
Graham: we talk to the breeders and they want to see the genes, the parents
Graham: GEMS will be the answer to cross-crop studies
Discussion on GEMS (Shawn)
Shawn shows the Workbook template for loading genes.
Ed: Gene reference should link to endnote
Put reference to Biblio
We also have the genebank sequence
DMS->Gems -> other DBs, so to do cross crop studies.
What to determine: What is the allelic substitution effect, Quantity, quality
If you can capture this distinction, then this is better
So were getting now in the cross detector
Mind the phenotypic and gene, in breeding
The beauty is what cannot be done in gems can be done in DMS
Are we not gonna put the gene in GEMS
Graham: Weng and Arllet, you should think of putting the gene in GEMS
Shawn: I want to still separate the genotypic and phenotypic studies. There should also be Local and central gems..the reason is that we want some data shown, but some not
If we are both putting data to the Central it will be a nightmare to Jesper or Arllet.
G talks about speed of development
Break out sessions
How to make IRRI-GRIMS generic(Bibiana)
Proposed workflow was presented. And all the steps for each workflow were identified for IRRI, WARDA, CIMMYT
G:: broad campaign.it is great but where are the resources. Are there resource?
R: no, we have each some priorities.
G: what is the most impt priority
CIMMYT: integrated: seed acq, multi, distri
G: it is big, you must break it in manageable size
R: specialized setgen for handling seed
G: breakdown the need, and prioritize
R: do we all have priority?
G: you should agree
R: what about WARDA
Tchamba: we apply ISO standard,
Short-term plans weren’t discussed yet as to how ICIS could help in generalizing Genebank activities
Long term plans were discussed
R: Priorities of GRC are to make grims more icis-oriented
Develop sort of a subset of setgen for handling incoming seeds could be the strategy for this year for grc
Management of IWIS3(Arllet)
- Release a definitive version by April, 2009 with version IWIS3_v5.5.-2009-002 in MySQL 5.??
- Some issues to check
- For method 31, check that GPID1<>GPID2 but GPID2=0
- Missing ATYPE or NTYPE from UQ/AAFC in the UDFLDS
- Location records
- Derivative lines, with name type Cross Name. This will be corrected next year after initial implementation of IWIS3 in the CIMMYT breeding
- Missing germplasm records to represent the lines which are sources of the imported lines in CIMMYT. The CIMMYT imported lines directly link to the cross. These records will be added next year
- DMS Trait, Scale and Method to be standardized
- Upload of project or institute-specific data to central IWIS3
- • 10M – 19,999,999 : UQ
- • 20M – 29,999,999 : AAFC
- • 30M – 39,999,999: India/AUS Project
- • 40M - ICARDA
- 4. Cookbook to be created in cropwiki about how to enter pedigrees in IWIS3
- 5. Zaid and Arllet to be added in the cropforge IWIS Communication
Arllet: to release a definitive IWIS 3
Issue of having a derivative line having a cross name;
DMS-trait-factor
Release 2x a year of local databases
Start a cookbook in wiki for entering pedigrees in iwis3
Start a communication in cropforge
Missing gids will be dealt with next year
Warehouse table issues(Ching)
Faster queries by arranging data – less tables, etc.
Cropfinder is using a data warehouse
Martin and mylah’s team are using DW
Thomas talked abt. Pros and cons
Complications in querying across studies
Thomas said will document the test results in cropwiki
Make queries that are faster
Cropfinder – using Data warehouse
Tables proposed by thomas
Thomas, will discuss what is being tested, and implemented in the iciswiki
Crop registries (Graham)
Graham: technical details in adding records
Tom: Look at new use cases
G: A tool that visualizes a management neighborhood like martin’s
Discussed abt. Public goods to be available in the central
Question of linking out to diff. genebanks
Web services as a tool to get data (Malaysian example)
Details on linking those that may be wrongly linked
Looked tools that are the available now
Have you discussed if this is single or many DBs? Malaysia trick. These are public data that we concentrated on.. Genebank data. We had pilot study on that..web integrated technique..
How to make ICIS more user-friendly (Ed)
- be workflow-driven
- efficient or effective?
- be configurable to user needs
Mimic breeding logic by making fewer steps only (customizable guis)
R: it is not supposed to be ‘user-friendly’ term but how to put data that are accurate.
R: All abt. Getting good quality data (and not to avoid mistakes thru functions)
Sophie: this is one aspect of user friendliness. We have to spend time to talk with breeders. Should be user friendly for breeders to make them confident to use it; and help them in their work
Ed: limited choices without reducing intituitiveness.
Martin: are we talking separate issues: our app is giving what they want OR how they want.. should we look at the 1st want?
User requirements: How to do it, to fit the Look and feel.
Martin: fulfills the breeder’s workflow even in 15 clicks; would like to get formal set of requirements; then how to do it to fit the user’s GUI;
Thomas: your talking abt. effectiveness and efficiency; use cropforge for feature requests
Ed: breedrs will use from launcher to workbook; all functions are there but there’s a problem in integration of all the icis tools;
Shawn: genebank could be a model for the workflows of breeders
Graham <word with memo template>; workflow based is the way to go; mistake of the 1st generation with liking to do so much; we try 1 tool do too much, we have abstracted the task. This is the wrong order.
of want to reverse the step in the next gen; need formal docu on what the breeders want to do or use cases;
Every breeder has a diff. naming convention and would like to capture these; customize and generalized in some level; describe the use cases and what could be generalized and customized;
Cam: warren’s workbook seems to have started going in that direction
G: yes but not sure if the menus are from setgen’s point of view or user’s
Ed:
Casper: need to think abt. Flexible systems
Ed: pipeline nurseries; processes are fixed and would not change much;
G:
Aaron: How to translate a breeder workflow to the system or basic design concepts; one aspect for lack of internal consistency;
Cam: when I installed ICIS, it did not work, so I have to ask..
There should be questions to ask to set up.
Cam: having problems with INI files; series of questions and not configure it as is;
Ed: if the program is modularized then just customized some menus for those modules;
Fran: ur saying u want breeders to have customized procedures…. And not like to set it up again and again; a breeder does not need to see all functionalities that he doesn’t need; could be within the crop, config is consistent;
Fran: additional customization for diff…. breeding location
Thomas: are there systems that exist that is configurable???
Ed: yes…
Thomas:
Martin: macros can be put together…. ;
Graham: that’s what workbook does….
Shawn: doesn’t want Microsoft based programs
Ed: have SOPS for diff. nurseries; went back to excel; it’s the breeder’s flowchart; map breeding process; visualize it; if we show them the process, and link them to the programmer’s code/functionalities;
Graham: did u draw a workflow… ?
Aaron: not yet.. ongoing…
Sofie: workflow is clear… on the way to define subtask of these workflows;
Martin: everyone would like to see some breeding workflows; its easier for us to see and
Sofie: let breeders do their own workflows…etc., sol’n just have to agree what are the priorities;
Ed: takes awhile for “baby sitting”; strength is icis understands the biological something…. But not having the workflow of the breeders…
Bibiana: final tests then user-friendliness
Shawn: im thnking abt. The current tools and what can be done now… set some ini keys and some functionalities are displayed and go away;
Graham: installation program is needed for every single OS
Shawn:
Graham:
Shawn: not separate the technical from the practical; put/document in wiki; breeders has to take responsibility; should read tutorials, etc.;
Mylah: quick iterations should be done and be in contact to the real breeders;
Thomas: who would do the config; is it the breeders or app. Managers???
Ed: app. Managers
Ed: breeding op. is
Thomas: app. Managers should do read the manuals
Bb: wiki for programmers; and a separate for the app.manager/breeders
Ed: inconsistency in in track and setgen’s headers
Sofie: should not be an issue if functions are customized;
Graham: arrange documents by workflow and not by applications
Ed: do u have programs that can do workflows ???
Martin: yes they are but ….. should connect bigger components…. Etc…; have workflow programs but not totally convinced \
G: there are programs in the chemical industries such as those
Ed:
Aaron: breeding canvass…. Modularized icons that represent functions; little components that can drag and drop and would create breeders’ own workflows;
Graham: QUGene is a simulation tool that understand a genetic model, breeding workflow; specs are hard for QUGene; develop a series of tools that ask the questions for the breeders; and it creates the workflow for the breeders by one lcik of a button;
Aaron: breeding program compiler….
Graham: save under a newname
Aaron: give users some authority as well to create these…
May: likes that approach; define even the labeling; customize even the labels;
Sofie: for breeding managers but not for end users;
Aaron:
Sofie: select appropriate building blocks are biased??
Zaid: based on the role, you’;ll see diff. menus; specific requirement for diff. users
Casper:
Zaid: to get actual user feedback in a systematic way; to understand each other better;
Cam: Some workflows
Martin: question of resources; implement workflows in the new gen; how much will be implemented in the current apps;
Ed:
May: same icis system background but users are given diff. perspective/displays
Ed: agrees with may
Sofie: value of workflow comes from the discussion bet. Breeders and app. Manager
Martin: modeling the process like using UML tools;
Ed:
Martin: lets use tools for modeling these processes;
Cam:
Martin: create some tool to formalize some sort of communication for feedbacks from diff. users;
Shawn: can workflows be shared?
Ed: yes but only the general ones
Sofie: Next step?
Fran: consistency of column orders of setgen
Future Planning
- ICIS Database Structure
- (CRIL) – Changes table for DMS (Arllet, Ching, Warren)
- (CRIL) – Identify maintenance methods in the Germplasm table (GNPGS=-2) (ICIS Developers)
- (CRIL) – Additional attributes (like Remarks to be saved in DMSAttr) for list entry (Candy, William)
- (CRIL) – Further Testing of PostgreSQL backend implementation for Central and Local databases (Ching, William, Warren)
- (CRIL/GRC) - Location Module (Grace, Ching)
- (CRIL) Location table; extra field for LOCID of "nearest named place"
- (CRIL) GEOREF table; extra fields for Source, Datum and Display precision
- (CRIL/GRC) Link of Location and Institute tables (Grace)
- (CRIL/GRC) tables for managing mission information (Grace, Ruaraidh, Beth)
- (GRC/CIMMYT) Joint review of ICIS & GRIN Schemata to identify points of cross linkage (Grace, Bibiana)
- (CRIL) Ontology (Martin, Mylah++, Rosemary, Arllet, Graham, Richard)
- GEMS (depending on available staff to develop software application)
- Web ICIS (Mylah++, Martin, Thomas)
- Local and Central concept in web koios
- Implement write access to database
- Implement basic authentication in web
- Implement connection to R modules
- Deployment of Koios (Weng, Jay for testing)
- Deployment of MySQL database (Weng, Manoj)
- WARDA, Africa (Tchamba)
- Deployment of MySQL database (Weng, Manoj)
- Warehouse tables
- Concrete applications:
- Cropfinder
- DArT Data
- Maintenance of Germplasm and study browser
- Improve list management in wheat data
- Implement one or two sub workflows (Breeding use cases)
- Pedigree/neighborhood viewer/curation/editing tool
- Cropfinder
- ICIS Database Access
- (CRIL/AAFC) Multi-user Access and Restriction in Postgres (Shawn,Ching)
- Security of data
- (GRC/CRIL)GRIMS Backend conversion of Oracle to PostgreSQL (not immediate)
- (CRIL/AAFC) Multi-user Access and Restriction in Postgres (Shawn,Ching)
- Stand-alone Applications
- (CRIL)-cross finding facility in standalone applications (Beth, Graham, Candy)
- (CRIL)- administrator’s tool (Ching, Weng, William)
- Loading/Deleting Central Studies
- Partial upload of Germplasm records
- Synchronization of GMS and DMS GIDs
- (CRIL/AAFC)-implementation of naming conventions (Candy, Shawn, Arllet)
- (CRIL) - Improvement of InTrack application (new staff)
- (CRIL/GCP) - Development of MoSel (Guy, Mylah++)
- (CRIL/CIMMYT)-finalizing version 8 of Fieldbook with links to ICIS (Vivek, Hector, Jean)
- (CRIL/INGER) - Seed ordering tool (Teri, Ed, Ching)
- (GRC) Enhancement of GRIMS - Location, People, Institute
- (CRIL/GRC) Incoming seeds (Beth, Martin, data managers of IRRI)
- (IRRI- GIS) GPG2 Location data curation and validation (Robert Hijmans)
- (CRIL) ICIS lite (Warren, Jean, Hector)
- Content Management/Documentation
- (ICIS community)-Breeding wiki/multimedia and workflow (Thomas)
- Rice (William)
- Cookbook for wheat (Arllet, Shawn, Sandra)
- Maize (Hector, Jean)
- Breeding training/multimedia materials (Thomas, Vivay)
- Review of the ICIS wiki/Re-organize contents (for next ICIS workshop)
- (ICIS community)-Breeding wiki/multimedia and workflow (Thomas)
- ICIS implementations and conversion
- (CRIL/PBGB/INGER/Bayer) Generalization of the management neighborhood (William, Victor, May)
- (CIMMYT/CRIL)-look at the transfer of wheat genetic resources data to ICIS following with the experience on rice (Arllet, Bibiana)
- (IRRI/CIAT/WARDA/IITA/GRIN)-GPG2 cross referencing of rice germplasm (Ruraidh, Tom H.)
- (CIMMYT/ICARDA/GRIN)-GPG2 cross referencing of wheat germplasm (Tom P, Tom H., Arllet)
- (UQ/AAFC/ICARDA/CRIL) – Uploading pedigree updates into IWIS3 and IBIS (Shawn, Sandra, Arllet)
- (GRC/CRIL) Validation of IRIS data (Ruaraidh, Grace, Victor, William, Tom H.)
ICIS Workshop 2010: Perth, Australia
- 1st or 2nd week of March 2010
- Contact person: Cameron Beeck of Canola Breeders Western Australia
REGISTERED PARTICIPANTS
| No | Name | Affiliation |
| 1 | Shawn Yates | Agriculture and Agri-Food Canada, Canada |
| 2 | Fran Clarke | Agriculture and Agri-Food Canada, Canada |
| 3 | Sophie Rozenfeld | Bayer BioScience - Bioinformatics, Belgium |
| 4 | Aaron Windsor | Bayer BioScience - Bioinformatics, Belgium |
| 5 | Sebastien Frade | Bayer BioScience - Bioinformatics, Belgium |
| 6 | Elicia Chew | Bayer BioScience - Rice, Singapore |
| 7 | May Ann Sallan | Bayer BioScience - Rice, Singapore |
| 8 | Ed Roumen | Bayer BioScience - Rice, Singapore |
| 9 | Cameron Beeck | Canola Breeders Western Australia, Australia |
| 10 | Bibiana Espinosa | CIMMYT, Mexico |
| 11 | Arllet Portugal | CIMMYT, Mexico |
| 12 | Graham McLaren | GCP, Mexico |
| 13 | Zaid Abdul-Hadi | ICARDA, Syria |
| 14 | Maria Corina Habito | IRRI-CRIL, Philippines |
| 15 | Clarissa Pimentel | IRRI-CRIL, Philippines |
| 16 | Martin Senger | IRRI-CRIL, Philippines |
| 17 | Richard Bruskiewich | IRRI-CRIL, Philippines |
| 18 | Mylah Rystie Anacleto | IRRI-CRIL, Philippines |
| 19 | Thomas Metz | IRRI-CRIL, Philippines |
| 20 | Warren Constantino | IRRI-CRIL, Philippines |
| 21 | Lilibeth Sison | IRRI-GRC, Philippines |
| 22 | Grace Capilit | IRRI-GRC, Philippines |
| 23 | Ruaraidh Sackville Hamilton | IRRI-GRC, Philippines |
| 24 | Ma. Concepcion Lotho | IRRI-PBGB-INGER, Philippines |
| 25 | Victor Alcantara | IRRI-PBGB-INGER, Philippines |
| 26 | William Eusebio | IRRI-PBGB-INGER, Philippines |
| 27 | Casper aan den Boom | Nunhems, Netherlands |
| 28 | Sandra Micallef | The University of Queensland, Australia |
| 29 | Tchamba Marimagne | WARDA, Benin, Africa |
| 30 | Thomas Hazekamp | IRRI Consultant, Italy |
| 31 | Violet Yong | IIR Exhibitions Pte Ltd, Singapore |
| 32 | Evylene Chua | IIR Exhibitions Pte Ltd, Singapore |
