license:cc-by-4.0 size_categories:n<1K format:parquet modality:text library:datasets library:pandas library:mlcroissant library:polars region:us
---
dataset_info:
  features:
  - name: dataset_id
    dtype: string
  - name: dataset_name
    dtype: string
  - name: source_url
    dtype: string
  - name: supported_entities
    sequence: string
  - name: supported_events
    sequence: string
  - name: description
    dtype: string
  - name: last_updated
    dtype: timestamp[ms]
  - name: coverage_timeframe
    dtype: string
  - name: created_at
    dtype: timestamp[ms]
  splits:
  - name: train
    num_bytes: 3419
    num_examples: 12
  download_size: 8586
  dataset_size: 3419
configs:
- config_name: default
  data_files:
  - split: train
    path: data/train-*
license: cc-by-4.0
---

# Crisis Prediction Datasets
## Dataset Card for dwb2023/crisis_prediction_datasets

### Purpose
A catalog of datasets supporting crisis prediction and response analysis, with particular focus on transportation and supply chain disruptions. This catalog tracks datasets that can be used to model temporal dependencies, test counterfactual scenarios, and analyze cascading failures in crisis situations.

### Intended Use
- Primary Use: Research and evaluation of crisis response datasets
- Intended Users: Data scientists, data engineers, and ML engineers working on crisis prediction systems
- Out of Scope: Real-time data ingestion or operational deployment

### Schema Description
The catalog uses the following schema to track and document datasets:

| Field Name | Data Type | Description |
|------------|-----------|-------------|
| dataset_id | string | Unique identifier for the dataset in the catalog (e.g., "OPENSKY_2020", "GDELT_2020_Q1") |
| dataset_name | string | Human-readable name of the dataset (e.g., "OpenSky Network", "GDELT Event Database") |
| source_url | string | URL or reference point for accessing the dataset |
| supported_entities | sequence[string] | List of entity types contained in the dataset (e.g., ["Airport", "Flight", "Policy"]) |
| supported_events | sequence[string] | List of event types the dataset can track (e.g., ["FlightCancellation", "HubClosure"]) |
| description | string | Detailed description of the dataset's contents, format, and relevance to crisis prediction |
| last_updated | timestamp[ms] | When the dataset itself was last updated, in milliseconds since epoch |
| coverage_timeframe | string | Time period covered by the dataset (e.g., "2020-03-20 to 2020-03-27") |
| created_at | timestamp[ms] | When this catalog entry was created, in milliseconds since epoch |

### Example Entry
```sql
{
    "dataset_id": "OPENSKY_2020",
    "dataset_name": "OpenSky Network",
    "source_url": "https://opensky-network.org/api",
    "supported_entities": ["Airport", "Flight"],
    "supported_events": ["FlightCancellation", "FlightDelay"],
    "description": "Real-time & historical ADS-B flight tracking data",
    "last_updated": "2025-01-30",
    "coverage_timeframe": "2020-03-20 to 2020-03-27",
    "created_at": "2025-01-31"
}
```

### Key Features
- Tracks datasets supporting temporal graph analysis
- Maps datasets to specific entity and event types
- Documents coverage periods aligned with crisis events
- Enables dataset comparison and selection for analysis

### Usage Caveats
- Catalog is for research planning purposes only
- Does not include actual data storage or automated ingestion
- Focus is on iterative research evaluation of datasets

### Implementation Details

#### Query Examples
```sql
-- Find datasets containing flight data
SELECT dataset_name, source_url 
FROM dataset_catalog 
WHERE supported_entities LIKE '%"Flight"%';

-- Get datasets covering March 2020
SELECT dataset_name, description 
FROM dataset_catalog 
WHERE coverage_timeframe LIKE '%2020-03%';

-- List recently updated datasets
SELECT dataset_name, last_updated 
FROM dataset_catalog 
ORDER BY last_updated DESC;
```

#### Key SQL Operations
```sql
-- Create new dataset entry
INSERT INTO dataset_catalog (
    dataset_id, dataset_name, source_url,
    supported_entities, supported_events,
    description, coverage_timeframe
) VALUES (
    'GDELT_2020_Q1',
    'GDELT Event Database',
    'https://www.gdeltproject.org/',
    '["Airport", "Policy", "Organization"]',
    '["PolicyChange", "BorderClosure"]',
    'Global event database tracking media-reported events',
    '2020-01-01 to 2020-03-31'
);

-- Update dataset coverage
UPDATE dataset_catalog
SET coverage_timeframe = '2020-01-01 to 2020-06-30'
WHERE dataset_id = 'GDELT_2020_Q1';
```

### Citation
This catalog was developed as part of research into crisis prediction and response systems, focusing on the March 2020 COVID-19 disruptions to global air cargo operations.