Skip to content

[19.0] fs_attachment: force_storage() re-uploads database-resident attachments for zero net effect, leaking one orphaned object per row per run #659

Description

@pimzand

Module: fs_attachment
Branches affected: verified on 19.0 (19.0.1.0.0) against the pinned source; the mechanism (_force_storage_to_object_storage + the force-db rules re-applied by the datas inverse) is present on 16.0 through 18.0 as well.

Describe the bug

ir.attachment.force_storage() selects every attachment whose store_fname is not on the object storage. That domain includes database-resident rows (store_fname NULL, content in db_datas), which on a stock database number in the hundreds to thousands: asset bundles and small images that the default force_db_for_default_attachment_rules ({"image/": 51200, "application/javascript": 0, "text/css": 0}) deliberately keep in the database.

For each such row it does attachment.write({"datas": attachment.datas}). The write uploads the content to the object storage, after which the datas inverse re-applies the force-db rules and stores the content straight back into db_datas. The result of each run:

  1. Zero net effect for those rows: the before/after distribution is identical (verified: same counts on storage and in the database, store_fname and fs_storage_id agreeing on every row).
  2. One orphaned object per row per run: the upload is never cleaned up when the row is re-homed to the database. The bucket accumulates unreferenced objects on every invocation.
  3. Runtime inflated by minutes of pointless upload work, which matters when force_storage() runs inside a maintenance window or a deployment pipeline.

Observed in production

On an instance with ~1300 database-resident attachments and 147 legitimately storage-resident ones, each force_storage() run took about six minutes, changed nothing, and left ~1300 new orphans. A bucket audit after several routine runs found 739 orphaned objects (~23 MB) against 147 referenced ones; every reference resolved, nothing was broken, but the garbage grows on every run.

A related side effect of the same walk: the write({"datas": ...}) round trip passes through the mimetype re-guess reported in #658, which corrupted text/css asset mimetypes and took a production site's styling down. The two issues are independent (fixing one does not fix the other) but share this code path as the trigger.

To Reproduce

  1. Configure an fs.storage backend as default for attachments, with the default force_db_for_default_attachment_rules.
  2. Use any database with normal content (asset bundles and small images exist in db_datas).
  3. Note the object count in the backend and the attachment distribution (store_fname NULL vs scheme-prefixed).
  4. Run env["ir.attachment"].force_storage() twice.
  5. Observe: distribution unchanged, runtime proportional to the number of database-resident rows on both runs, and the backend object count grown by roughly that number per run.

Expected behavior

force_storage() should not touch rows whose storage decision will put them straight back where they already are. Database-resident attachments that match the force-db rules are already in their configured location; walking them is pure churn.

Suggested fix

Exclude them at selection time: subtract _store_in_db_instead_of_object_storage_domain() from the search domain in _force_storage_to_object_storage() (the domain already exists and is documented as mirroring the decision logic). Alternatively, short-circuit per row before uploading when _store_in_db_instead_of_object_storage(data, mimetype) is true. Either variant makes repeated force_storage() runs idempotent in both effect and cost, and stops the orphan growth.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions