Workflow dashboard

Main Page > Vancouver Digital Archives > Software Integration/Development

Workflow dashboard tool
Digital Archives Workflow Event Tracker (WETboard!): (should come up with a better name so that it has a catchier acronym)

There should be a tool that tracks each SIP through the ingest process (this has expanded to all events in the system, not just Ingest events). I see this as a database that contains administrative metadata about the ingest process all events, and an interface that allows the archivist to query the database to view the status of SIPs/AIPs, and potentially an interface that allows the archivist to select SIPs/AIPs and initiate processes on them. We may also want some code that restricts certain actions if it is likely to cause the system to choke-up
 * When SIPs are received by the archives, SIPs are registered on the WETboard
 * As SIP progresses through the ingest process all event stages, the SIP record on the WETboard is updated to reflect what stage in the process the SIP is at, did specific processes run properly without crashing, summary results of inspection events (integrity checks, malware checks, etc.), the actual location of the SIP, the AIP or their components, type, name and location of reports produced, messages that have been sent to RIM or other creators (or users?) about SIP progress, etc
 * Some of the information will be explicitly present in the metadata that is produced during ingest, some of it will be implicit in the metadata, and some will be implicit by the fact that some metadata exists. I think there should be a way of pulling all of this together so that the archivist can see what is going on, rather than having to look into several logs
 * I put together a table with some of the WETboard metadata elements I would expect to see. This should be taken as an illustrative example of the types of things this tool should be tracking, rather than a legit attempt to sketch out the metadata.
 * These will likely correspond to PREMIS event entities that are yet to be defined (attempts are being made here:

see also Metadata entity analysis

EXAMPLE
{| border="1" cellpadding="10" cellspacing="0" width=90% Etc.
 * - style="background-color:#cccccc;"
 * Element
 * Description/Purpose
 * Data Type
 * SIP_ID
 * Unique ID of the SIP
 * string
 * Transfer_ID
 * Unique ID of transfer SIP was attached to
 * string
 * TransferDate
 * Time SIP was placed in network share
 * date/time
 * SIPtransfer_location
 * Location the received SIP was transferred to
 * Network path (e.g., /home/demo/sharedfolder/[SIPname])
 * SIPbackup_location
 * Location that the back-up copy of the received SIP was transferred to
 * Network path (/othermachine/[SIPname]
 * SIP_metadata
 * Network path of the .xml metadata in the SIP
 * Network path (e.g., /home/demo/sharedfolder/[SIPname]/[SIPname].xml)
 * SIP_file_count
 * Count of the total number of files included in the received SIP
 * Integer
 * SIP_contentObject_count
 * Count of the total number of content objects included in the received SIP
 * Integer
 * Current_State
 * Name of the process currently running on the SIP
 * {IngestQueue, Audit, AppraisalQueue, NormalizationQueue, Normalization, StorageTransferQueue, etc.}
 * Receipt_acknowledged
 * Has message been sent to RIM acknowledging receipt of SIP in transfer folder
 * {yes, no}
 * Audit_Start
 * Time audit process was initiated
 * Date/time
 * Audit_End
 * Time audit ended
 * Date/time
 * Manifest_Check_success
 * Did the manifest check run properly? (i.e., "did it run without crashing?", not "was the result of the check positive?")
 * {yes, no}
 * Manifest_Check
 * Result of manifest check
 * {pass, fail}
 * Metadata_Check_success
 * Metadata_Check
 * Result of metadata check
 * {pass, fail}
 * Integrity_Check_success
 * Integrity_Check
 * Result of md5 integrity check
 * {pass, fail}
 * Begin_Quarantine
 * Time SIP entered quarantine
 * Date/time
 * End_Quarantine
 * Time SIP exited quarantine
 * Date/time
 * Malware_Check
 * Result of Malware check
 * {pass, fail}
 * Files_inspected
 * Count of all files inspected by malware check
 * Integer
 * Clean_files
 * Count of all files with no malware present on inspection
 * Integer
 * Repaired_files
 * Count of infected files that were successfully repaired
 * Integer
 * Infected_files
 * Count of infected files that were not repaired
 * Integer
 * Isolation_location
 * Network location infected files were moved to
 * Network path (e.g., /home/demo/malwareIsolation/[SIPname]/)
 * Audit_Result
 * Result of the audit
 * {Pass, Fail, conditionalPass(?)}
 * Audit_Log
 * Network location of the audit log
 * RIM_auditNotification
 * Has message been sent to RIM notifying of audit results
 * RIM_resubmissionRequest
 * Has RIM been sent a resubmission request
 * Accession_ID
 * Accession ID assigned to SIP
 * Normalized
 * Has SIP content been normalized
 * Normalization_report
 * Location of normalization report
 * -}
 * Clean_files
 * Count of all files with no malware present on inspection
 * Integer
 * Repaired_files
 * Count of infected files that were successfully repaired
 * Integer
 * Infected_files
 * Count of infected files that were not repaired
 * Integer
 * Isolation_location
 * Network location infected files were moved to
 * Network path (e.g., /home/demo/malwareIsolation/[SIPname]/)
 * Audit_Result
 * Result of the audit
 * {Pass, Fail, conditionalPass(?)}
 * Audit_Log
 * Network location of the audit log
 * RIM_auditNotification
 * Has message been sent to RIM notifying of audit results
 * RIM_resubmissionRequest
 * Has RIM been sent a resubmission request
 * Accession_ID
 * Accession ID assigned to SIP
 * Normalized
 * Has SIP content been normalized
 * Normalization_report
 * Location of normalization report
 * -}
 * RIM_resubmissionRequest
 * Has RIM been sent a resubmission request
 * Accession_ID
 * Accession ID assigned to SIP
 * Normalized
 * Has SIP content been normalized
 * Normalization_report
 * Location of normalization report
 * -}
 * Has SIP content been normalized
 * Normalization_report
 * Location of normalization report
 * -}
 * Location of normalization report
 * -}
 * -}