Zenodo Dataset Downloader

A simple and easy-to-use .NET tool for high-speed downloading of Zenodo.org datasets, just provide the DOI.

Features

• ✅ Easy to use: Just provide the DOI to download
• ✅ High-speed download: Supports multi-threaded concurrent download and chunked multi-threaded download
• ✅ Chunked download: Large files (>10MB) are automatically chunked and downloaded concurrently with multiple threads, significantly improving speed
• ✅ Auto parsing: Automatically extracts record information from DOI
• ✅ Progress display: Real-time download progress display
• ✅ Resume support: Supports HTTP Range requests, can automatically resume after interruption
• ✅ Auto retry: Automatically retries on network errors, up to 5 retries, ensuring successful download
• ✅ Smart detection: Automatically detects existing files, skips complete files, resumes incomplete files
• ✅ Pure .NET implementation: No additional dependencies required
• ✅ Modern GUI: User-friendly Windows desktop application with intuitive interface

Usage

Build the project

# Debug build
dotnet build

# Release build (optimized)
dotnet build -c Release

After building, the executable will be located at:

• Debug: bin\Debug\net8.0-windows\ZenodoDownloader.exe
• Release: bin\Release\net8.0-windows\ZenodoDownloader.exe

Run the application

After building, run the executable:

# Windows
.\bin\Debug\net8.0-windows\ZenodoDownloader.exe

# Or use dotnet run
dotnet run

Using the GUI

1. Launch the application: Run ZenodoDownloader.exe to open the graphical interface.

2. Enter DOI:

   • In the DOI input field, enter the Zenodo dataset DOI (e.g., 10.5281/zenodo.8200423)
   • You can use either the full DOI or just the record ID

3. Start download:

   • Click the "Start Download" button
   • A folder browser dialog will appear - select the directory where you want to save the files
   • Click "OK" to start the download

4. Monitor progress:

   • Overall Progress: Shows the progress of all files being downloaded (files completed/total files and percentage)
   • Current File Progress: Shows the progress of the file currently being downloaded with file size and speed
   • Chunk Download Progress: For large files (>10MB), shows the progress of each chunk being downloaded
   • Status Log: Displays detailed status messages and download information

5. Cancel download (if needed):

   • Click the "Cancel" button to stop the current download
   • Partial downloads will be saved and can be resumed later

Example Usage

Example 1: Download a dataset

1. Launch ZenodoDownloader.exe
2. Enter DOI: 10.5281/zenodo.8200423
3. Click "Start Download"
4. Select download directory: C:\Downloads\MyDataset
5. Wait for download to complete

Example 2: Resume interrupted download

1. Launch ZenodoDownloader.exe
2. Enter the same DOI as before
3. Click "Start Download"
4. Select the same directory where partial files exist
5. The tool will automatically detect and resume incomplete files

Application Icon

The application uses a custom icon (downloader.ico) that appears in:

• The executable file icon
• The window title bar
• The taskbar when the application is running

System Requirements

• Windows operating system
• .NET 8.0 or higher runtime
• Internet connection for downloading datasets

How It Works

1. Extract record ID (e.g., 8200423) from DOI (e.g., 10.5281/zenodo.8200423)
2. Call Zenodo API to get dataset metadata and file list
3. Check local file status:
   • If file is complete, automatically skip
   • If file is incomplete, use HTTP Range request to resume from breakpoint
   • If file doesn't exist, download from the beginning
4. Intelligently select download strategy:
   • Large files (≥10MB): Automatically chunked, use multi-threaded concurrent download for each chunk, then merge
   • Small files (<10MB): Single-threaded download, simple and efficient
5. Use multi-threaded concurrent download for all files
6. Display download progress and statistics in real-time

Chunked Multi-threaded Download Feature

The tool supports intelligent chunked multi-threaded download, significantly improving download speed for large files:

• Auto chunking: Automatically enabled for files ≥10MB
• Dynamic chunk count: Automatically calculates optimal chunk count based on file size (up to 8 chunks)
• Concurrent download: All chunks download simultaneously, fully utilizing bandwidth
• Chunk-level resume: Each chunk supports resume, only resumes incomplete chunks after interruption
• Auto merge: Automatically merges all chunks into complete file after download completes
• Temporary file management: Uses temporary directory to store chunk files, automatically cleaned up after merge

Performance advantages:

• Large file download speed can be improved by 3-8x (depending on network conditions)
• Fully utilizes multi-core CPU and network bandwidth
• Each chunk downloads independently without affecting others

Resume Feature

The tool supports complete resume functionality:

• Auto detection: Checks existing files in output directory on startup
• Smart resume: If file is incomplete, automatically uses HTTP Range request to continue from breakpoint
• Chunk-level resume: When chunked downloading, each chunk can be independently resumed
• Progress display: Shows downloaded size, total size, completion percentage and remaining size when resuming
• Server compatibility: If server doesn't support Range requests, will automatically download from beginning and prompt

Example scenarios:

• Network interruption during large file download, rerunning program will automatically continue from breakpoint
• When chunked downloading, completed chunks will be skipped, only incomplete chunks will be resumed
• When downloading multiple files, completed parts will be skipped, only incomplete parts will be downloaded

Auto Retry Feature

The tool has built-in powerful auto retry mechanism, ensuring successful download even when network is unstable:

• Smart retry: Automatically retries when encountering network errors (such as connection interruption, EOF errors, transport stream errors, etc.)
• Retry strategy: Up to 5 retries, 2 seconds delay between each retry
• Error detection: Automatically identifies retryable error types (network exceptions, IO exceptions, HTTP request exceptions, etc.)
• Resume retry: Retry automatically continues from downloaded position, won't re-download completed parts
• Chunk-level retry: When chunked downloading, each chunk has independent retry mechanism
• File verification: Automatically verifies file size after download completes, ensuring file integrity

Retry scenarios:

• Sudden network connection interruption (EOF errors, transport stream errors)
• Server temporarily unavailable
• Network timeout
• Other temporary network issues

Example output:

[Downloading] file.zip (100.5 MB)...
[Error] file.zip: Received an unexpected EOF or 0 bytes from the transport stream
[Retry] file.zip - Will retry after 2 seconds (1/5)...
[Retry Download] file.zip (100.5 MB) - Retry attempt 1
[Completed] file.zip (100.5 MB)

Interface Elements

The GUI provides the following information:

• DOI Input Field: Enter the Zenodo dataset DOI here (e.g., 10.5281/zenodo.8200423)
• Start Download Button: Initiates the download process (disabled during download)
• Cancel Button: Appears during download to stop the current operation
• Dataset Info: Displays the title of the dataset being downloaded
• Overall Progress Bar: Shows the overall download progress of all files with percentage and file count
• Current File Progress Bar: Shows the progress of the currently downloading file with file name, downloaded/total size, and percentage
• Chunk Progress Section: Appears for large files (>10MB), showing progress of each download chunk with individual progress bars
• Status Log: Displays detailed status messages including:
  • DOI parsing status
  • Record ID extraction
  • Dataset information retrieval
  • File discovery and size information
  • Download progress for each file
  • Chunk download status (for large files)
  • Merging progress (for chunked downloads)
  • Error messages and retry attempts
  • Completion status and time elapsed

Tips

• Large Files: Files larger than 10MB will automatically use chunked download for better performance
• Resume Downloads: If a download is interrupted, simply restart the application and download the same dataset to the same directory - it will automatically resume
• Multiple Files: The tool downloads multiple files concurrently, so you'll see progress for different files simultaneously
• Network Issues: The tool automatically retries failed downloads up to 5 times, so temporary network issues won't cause permanent failures

Troubleshooting

Problem: Download fails immediately

• Solution: Check your internet connection and verify the DOI is correct

Problem: Download is very slow

• Solution: This is normal for large files. The tool uses multi-threaded download to maximize speed, but actual speed depends on your network and Zenodo server

Problem: "Invalid DOI format" error

• Solution: Make sure the DOI contains "zenodo." (e.g., 10.5281/zenodo.8200423)

Problem: "File list not found" error

• Solution: The dataset may not exist or may be restricted. Verify the DOI is correct and accessible

Problem: Download stops at a certain percentage

• Solution: The tool will automatically retry. If it continues to fail, check your network connection and try again later

Notes

• Ensure sufficient disk space to store downloaded files
• Download speed depends on network connection and Zenodo server response speed
• Please comply with Zenodo's terms and conditions of use
• The application requires .NET 8.0 runtime to be installed on your system
• The application is a Windows-only desktop application (WPF)
• Progress tracking has been optimized to prevent exceeding 100% during concurrent downloads