Skip to content

Fix data import with S3 storage backend - #12973

Open
maxgr0 wants to merge 3 commits into
inventree:masterfrom
maxgr0:fix/importer-s3-binary-mode
Open

maxgr0 wants to merge 3 commits into
inventree:masterfrom
maxgr0:fix/importer-s3-binary-mode

Conversation

@maxgr0

@maxgr0 maxgr0 commented Oct 5, 2026 •

Copy link
Copy Markdown

First of all, very much appreciate what you guys are doing here! We're implementing InvenTree right now and will most probably come up with some ideas and PRs in the future as we can allocate some capacity for bringing this project forward :)

Problem

With media stored in S3, every data import fails on session creation with HTTP 500, also reported by another user in #11117.

File ".../importer/operations.py", line 36, in load_data_file
    file_object.open('r')
File ".../storages/backends/s3.py", line 153, in open
    raise ValueError("Cannot reopen file with a new mode.")

This affects all supported formats, because the failing line runs before any format specific code.

Cause

After DataImportSession.save(), self.data_file is a fresh FieldFile and data_file.file opens it through the storage backend in mode rb. load_data_file() then calls open('r') on it. When configuring S3, the S3File.open() raises when the requested mode differs from the one the file was opened with. Django's File.open() (local storage) only seeks to the start if the file is already open and ignores the requested mode, so the bug never shows locally.

Fix

Open the file in binary mode. The function already decodes text formats itself (utf-8-sig with cp1252 fallback) and passes Excel data to tablib as bytes, so binary is what the code expects. Behaviour with local file storage is unchanged.

Testing

After DataImportSession.save(), self.data_file is a fresh FieldFile and data_file.file opens it through the storage backend in mode rb. load_data_file() then calls open('r') on it. django-storages' S3File.open() raises when the requested mode differs from the one the file was opened with. Django's File.open() (local storage) only seeks to the start if the file is already open and ignores the requested mode, so the bug never shows locally.

Testing

Reproduced on InvenTree 1.5.6 (Python 3.14, django-storages 1.14.6) with media on real S3:

  • before: POST /api/importer/session/ with a csv and an xlsx both return 500 with the error above. The file is already uploaded, so each attempt leaves an orphaned session in state INITIAL plus its file in the bucket.
  • after: both return 201 with the columns extracted; accepting the field mapping extracts the correct rows. No error comes up. Existing importer unit tests (local storage) unaffected. Imported fields are all correct.

@netlify

netlify Bot commented Oct 5, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for inventree-web-pui-preview canceled.

Name Link
🔨 Latest commit 2556515
🔍 Latest deploy log https://app.netlify.com/projects/inventree-web-pui-preview/deploys/6ac4eae238ae4e00084c28c1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant