Ultra2 exports
At the moment the pipeline runs, imports to SQL and outputs everything to CSV/SQLite, plus safe haven exports. The u036 workspaces on Ultra2 will require exporting subsets to different places. Suggested implementation:
- import everything to SQL as normal
- decouple (or at least separate) export process, and split it up into different types of export
- full, nhs-safe-haven, u036-isaric, u036-phosp, u036-collab
- certain tables should only be present in certain spaces
- safe haven exports only wants patient ID/timepoint
- maybe we'll need to filter to certain patients?
Since the exports are built entirely from the database, the CSVs and docs should be easy enough. SQLite will require an option in #21 (closed) to export subsets.
Not sure how NGS data will be handled - if we point to it with file paths, will those file paths be accessible to the people who view the table?