Download and Use the Public Bundle

1. Clone and verify

git clone https://github.com/rayborg/dctabgan-ids-benchmark-datasets.git
cd dctabgan-ids-benchmark-datasets
shasum -a 256 -c SHA256SUMS.txt

The checksum manifest covers the 17 downloadable CSVs plus docs, metadata, and scripts. If you later recreate omitted tasks locally, verify those separately with scripts/verify_omitted_datasets.py.

python3 scripts/list_tasks.py --status downloadable
python3 scripts/export_benchmark_splits.py \
  --task friday_bot \
  --output-dir ml_exports/friday_bot

2. Understand the files

Each downloadable task is a full cleaned CSV, not a separate train/test split export. The benchmark definitions specify the split semantics, and the benchmark code reconstructs train/test partitions from the row order and counts. Every task has exactly 500 minority attack rows in train and 500 minority attack rows in test.

Already done: binary labels, selected rows, no-DoS scope, preserved-ratio counts, benchmark row order, and recorded materialization drops. Not done: scaling, one-hot encoding, train-only imputation, estimator-specific transforms, or separate train/test files. The full CSVs retain source_row_index for auditability; the split helper drops it by default.

Key file	What it tells you
`metadata/datasets.csv`	One-row-per-task summary including counts, provenance notes, and whether the CSV is downloadable.
`metadata/datasets.json`	Full machine-readable task manifest with benchmark paths and local reproduction details.
`30D benchmark definition`	The exact task list and processed dataset filenames.

3. Direct links and command-line use

For one-file downloads, use the raw CSV links in the catalog below. For a local manifest of paths and caveats, use the task listing helper:

python3 scripts/list_tasks.py --status downloadable --format csv

To export benchmark train/test files from any public task:

python3 scripts/export_benchmark_splits.py \
  --task 5g_nidd_tcp_connect_scan_vs_benign \
  --output-dir ml_exports/5g_tcp

python3 scripts/export_benchmark_splits.py \
  --task 5g_nidd_tcp_connect_scan_vs_benign \
  --output-dir ml_exports/5g_tcp_encoded \
  --encoded

The optional --encoded output uses basic train-only fitting for numeric fill values and categorical one-hot levels, then applies that encoder to test rows.

4. Corpus-level redistribution status for public CSVs

Corpus	Tasks	Redistribution status
CIC-IDS-2017	4	Downloadable with CIC provenance/citation caveat; no project relicensing.
CSE-CIC-IDS2018	4	Downloadable with CIC/UNB provenance/citation caveat; no project relicensing.
HIKARI-2021	2	Downloadable under documented CC BY 4.0 route with attribution.
5G-NIDD	3	Downloadable under Fairdata CC BY 4.0 route; official IEEE route remains gated.
RT-IoT2022	4	Downloadable under UCI CC BY 4.0 route with attribution.

5. Downloadable task catalog with direct CSV links

Task key	Corpus	Direct CSV	Redistribution status
`friday_bot`	CIC-IDS-2017	Raw CSV `data/cic-ids-2017/friday_bot/friday_bot_vs_benign_preservedratio_train500minority_48000benign_test500minority_48000benign_chrono.csv`	CIC provenance/citation caveat; no project relicensing.
`thursday_web_attack_bruteforce`	CIC-IDS-2017	Raw CSV `data/cic-ids-2017/thursday_web_attack_bruteforce/thursday_web_attack_bruteforce_vs_benign_preservedratio_train500minority_55500benign_test500minority_55500benign_chrono.csv`	CIC provenance/citation caveat; no project relicensing.
`tuesday_ftp_patator`	CIC-IDS-2017	Raw CSV `data/cic-ids-2017/tuesday_ftp_patator/tuesday_ftp_patator_vs_benign_preservedratio_train500minority_27000benign_test500minority_27000benign_chrono.csv`	CIC provenance/citation caveat; no project relicensing.
`tuesday_ssh_patator`	CIC-IDS-2017	Raw CSV `data/cic-ids-2017/tuesday_ssh_patator/tuesday_ssh_patator_vs_benign_preservedratio_train500minority_36500benign_test500minority_36500benign_chrono.csv`	CIC provenance/citation caveat; no project relicensing.
`cse_cic_ids2018_bot`	CSE-CIC-IDS2018	Raw CSV `data/cse-cic-ids2018/cse_cic_ids2018_bot/cse_cic_ids2018_bot_vs_benign_preservedratio_train500minority_23500benign_test500minority_23500benign_chrono.csv`	CIC/UNB provenance/citation caveat; no project relicensing.
`cse_cic_ids2018_ftp_bruteforce`	CSE-CIC-IDS2018	Raw CSV `data/cse-cic-ids2018/cse_cic_ids2018_ftp_bruteforce/cse_cic_ids2018_ftp_bruteforce_vs_benign_preservedratio_train500minority_34500benign_test500minority_34500benign_chrono.csv`	CIC/UNB provenance/citation caveat; no project relicensing.
`cse_cic_ids2018_infilteration`	CSE-CIC-IDS2018	Raw CSV `data/cse-cic-ids2018/cse_cic_ids2018_infilteration/cse_cic_ids2018_infilteration_vs_benign_preservedratio_train500minority_41500benign_test500minority_41500benign_chrono.csv`	CIC/UNB provenance/citation caveat; no project relicensing.
`cse_cic_ids2018_ssh_bruteforce`	CSE-CIC-IDS2018	Raw CSV `data/cse-cic-ids2018/cse_cic_ids2018_ssh_bruteforce/cse_cic_ids2018_ssh_bruteforce_vs_benign_preservedratio_train500minority_35500benign_test500minority_35500benign_chrono.csv`	CIC/UNB provenance/citation caveat; no project relicensing.
`hikari_bruteforce_vs_benign`	HIKARI-2021	Raw CSV `data/hikari-2021/hikari_bruteforce_vs_benign/hikari_bruteforce_vs_benign_preservedratio_train500minority_43500majority_test500minority_43500majority_modern7.csv`	Documented CC BY 4.0 route; attribution required.
`hikari_probing_vs_benign`	HIKARI-2021	Raw CSV `data/hikari-2021/hikari_probing_vs_benign/hikari_probing_vs_benign_preservedratio_train500minority_11000majority_test500minority_11000majority_modern7.csv`	Documented CC BY 4.0 route; attribution required.
`5g_nidd_tcp_connect_scan_vs_benign`	5G-NIDD	Raw CSV `data/5g-nidd/5g_nidd_tcp_connect_scan_vs_benign/5g_nidd_tcp_connect_scan_vs_benign_preservedratio_train500minority_11500majority_test500minority_11500majority_additional10.csv`	Fairdata CC BY 4.0 route; official IEEE route remains gated.
`5g_nidd_syn_scan_vs_benign`	5G-NIDD	Raw CSV `data/5g-nidd/5g_nidd_syn_scan_vs_benign/5g_nidd_syn_scan_vs_benign_preservedratio_train500minority_11500majority_test500minority_11500majority_additional10.csv`	Fairdata CC BY 4.0 route; official IEEE route remains gated.
`5g_nidd_udp_scan_vs_benign`	5G-NIDD	Raw CSV `data/5g-nidd/5g_nidd_udp_scan_vs_benign/5g_nidd_udp_scan_vs_benign_preservedratio_train500minority_15000majority_test500minority_15000majority_additional10.csv`	Fairdata CC BY 4.0 route; official IEEE route remains gated.
`rt_iot2022_nmap_udp_scan_vs_benign`	RT-IoT2022	Raw CSV `data/rt-iot2022/rt_iot2022_nmap_udp_scan_vs_benign/rt_iot2022_nmap_udp_scan_vs_benign_preservedratio_train500minority_2000majority_test500minority_2000majority_additional10.csv`	UCI CC BY 4.0 route; attribution required.
`rt_iot2022_nmap_xmas_tree_scan_vs_benign`	RT-IoT2022	Raw CSV `data/rt-iot2022/rt_iot2022_nmap_xmas_tree_scan_vs_benign/rt_iot2022_nmap_xmas_tree_scan_vs_benign_preservedratio_train500minority_3000majority_test500minority_3000majority_additional10.csv`	UCI CC BY 4.0 route; attribution required.
`rt_iot2022_nmap_os_detection_vs_benign`	RT-IoT2022	Raw CSV `data/rt-iot2022/rt_iot2022_nmap_os_detection_vs_benign/rt_iot2022_nmap_os_detection_vs_benign_preservedratio_train500minority_3000majority_test500minority_3000majority_additional10.csv`	UCI CC BY 4.0 route; attribution required.
`rt_iot2022_nmap_tcp_scan_vs_benign`	RT-IoT2022	Raw CSV `data/rt-iot2022/rt_iot2022_nmap_tcp_scan_vs_benign/rt_iot2022_nmap_tcp_scan_vs_benign_preservedratio_train500minority_6000majority_test500minority_6000majority_additional10.csv`	UCI CC BY 4.0 route; attribution required.

6. Load a dataset in Python

import pandas as pd

df = pd.read_csv(
    "data/cic-ids-2017/friday_bot/"
    "friday_bot_vs_benign_preservedratio_train500minority_48000benign_"
    "test500minority_48000benign_chrono.csv"
)

print(df.shape)
print(df["label"].value_counts())

The target column is always label. The public bundle uses 0 for the majority benign/normal class and 1 for the minority attack class.