IRODS

Main Page > Vancouver Digital Archives > Technology/Tools Evaluation > iRODS

Integrated Rule-Oriented Data System
IRODS is a framework for managing large scale data networks. At its core are a set of user developed rules, written in C that handle the way in which data stored, replicated, and accessed with in the grid. This allows for a great level of fine tuning. The following page is a overview of how this system may fit into a OAIS based system.

OAIS Requirements
Before getting into the process of writing IRODS rule sets we should have a basic idea of what OAIS archival storage requirements are met and which are not.

Open Source
irods is released under a BSD license. BSD is a very free license that allows use to modify, reuse, sell.. etc etc this fits with in OAIS requirements

Runs on 'simple' PC/Servers
irods requires no special hardware to run, and has been tested successfully on many *nix based systems including:
 * Linux (RedHat) 32-bit, with PostgreSQL
 * Linux (RedHat) 32-bit, with Oracle
 * Linux (RedHat) 64-bit, with PostgreSQL
 * Linux (Ubuntu) 32-bit, with PostgreSQL
 * Linux (SuSE) 64-bit, with PostgreSQL (Teragrid)
 * Solaris, 32-bit, with PostgreSQL
 * Solaris, 32-bit, with Oracle
 * Mac OS X, PowerPC, with PostgreSQL
 * Mac OS X, Intel 32-bit, with PostgreSQL
 * AIX, via non-ICAT server (DataStar)

Scalability
irods is designed as a storage grid. Over time as the more data is ingested into the system nodes can be added to meet increasing storage demand. However the process in which data is distributed does not appear to be default and new rule sets need be created in order to meet this need.

Redundant File Storage
Irods has a feature called "resource groups". Each irods server is a "resource" that can be added to a specific group Once this is done special client rules can be written to make sure new files are mirrored to all resources with in the group.

Managing Backups
One of Irods coolest features is the micro services that allow for data replication as new files are added. This could be used to replicate data over WAN. IE data can be ingested by your cluster in Vancouver and instantly replicated to your cluster in Toronto. However again this is based on microservices and require some level of programing to be done on the actual irods server. Alternatively If a high speed WAN link is not available data backup could be handled by a tape system(amanda?).

Communication with other storage systems
This is handled with irods federation rule sets. This allows seperate grids to share data while still maintaining control and security over their separate zones.

"Federation is a feature of iRODS 1.2 (and beyond) in which separate iRODS Zones (iRODS instances), can be integrated. When zones 'A' and 'B' are federated, they work together. "

Installation
Installing the irods system requires linux make tools. In Ubuntu you must add the package build-essential before moving on.


 * Download the source
 * Run ./irodssetup This will go ahead and ask you a number of questions reguarding the setup of your system .  for the intial tests all of the default options worked.

Running icommands
icommands are unix commands that manipulate the irods system filesystem. The commands run very similarly to there unix counterparts except when these are run they are run with in the world of the irods rule sets.

Unix-like commands

 * iinit Initialize - Store your password in a scrambled form for automatic use by other icommands.
 * iput Store a file
 * iget Get a file
 * imkdir Like mkdir, make an iRODS collection (similar to a directory or Windows folder)
 * ichmod Like chmod, allow (or later restrict) access to your data objects by other users.
 * icp Like cp or rcp, copy an iRODS data object
 * irm Like rm, remove an iRODS data object
 * ils Like ls, list iRODS data objects (files) and collections (directories)
 * ipwd Like pwd, print the iRODS current working directory
 * icd Like cd, change the iRODS current working directory
 * irepl Replicate data objects.
 * iexit Logout (use 'iexit full' to remove your scrambled password from the disk)
 * ipasswd Change your irods password.
 * ichksum Checksum one or more data-object or collection from iRODS space.
 * imv Moves/renames an irods data-object or collection.
 * iphymv Physically move files in iRODS to another storage resource.
 * ireg Register a file or a directory of files and subdirectory into iRODS.
 * irmtrash Remove one or more data-object or collection from a RODS trash bin.
 * irsync Synchronize the data between a local copy and the copy stored in iRODS or between two iRODS copies.
 * itrim Trim down the number of replica of a file in iRODS by deleting some replicas.
 * iexecmd Remotely Execute (fork and exec) a command on the server.
 * imcoll Manage (mount, unmount, synchronize and purge of cache) mounted iRODS collections and the associated cache.
 * ibun Upload and download structured (e.g. tar) files.

Metadata

 * imeta Add, remove, list, or query user-defined Attribute-Value-Unit triplets metadata
 * isysmeta Show or modify system metadata
 * iquest Query (pose a question to) the ICAT, via a SQL-like interface

Informational

 * ienv Show current iRODS environment
 * ilsresc List resources
 * iuserinfo List users
 * imiscsvrinfo Get basic server information; test communication.

Administration

 * iadmin Administration commands: add/remove/modify users, resources, etc.

Rules and Delayed rule execution

 * irule Submit a user defined rule to be executed by an irods server.
 * iqstat Show pending iRODS rule executions.
 * iqdel Removes delayed rules from the queue.
 * iqmod Modifies delayed rules in the queue.

Setting up Rulesets
**add info**