zcloud.console_scripts.scumover module

SCU Mover

This script provides a command-line interface (CLI) for transferring files based on a provided schema and directory. It uses the provided schema to discover files in the specified directory, generates a manifest, validates it, and uploads the files along with the manifest to a Google Cloud Storage bucket.

Functions

scu_transfer()

Imported Functions

  • SCUManifestSchemaHandler: A handler for SCU manifest schema validation.

  • run_discovery: Discovers files in a directory based on a given schema and generates a manifest.

  • send_files_from_manifest: Uploads files and the manifest to a Google Cloud Storage bucket.

Schemas

SCU_RAW.JSON

Defines the structure and validation rules for the SCU raw schema. It includes the following properties: - name: The name of the schema. - schema_version: The version of the schema. - discovery_rules: Rules for discovering files. - elements: Elements to be included in the schema. - compute_type: The type of computation to be performed.

SCU_MANIFEST.JSON

Defines the structure and validation rules for the manifest file. It includes the following properties: - files: List of files to be included in the manifest. - meta: Metadata associated with the manifest. - elements: Elements to be included in the manifest. - element_files: Files associated with each element.