Module: fs_attachment
Branches affected: verified on 19.0 (19.0.1.0.0) against the pinned source; the mechanism (_force_storage_to_object_storage + the force-db rules re-applied by the datas inverse) is present on 16.0 through 18.0 as well.
Describe the bug
ir.attachment.force_storage() selects every attachment whose store_fname is not on the object storage. That domain includes database-resident rows (store_fname NULL, content in db_datas), which on a stock database number in the hundreds to thousands: asset bundles and small images that the default force_db_for_default_attachment_rules ({"image/": 51200, "application/javascript": 0, "text/css": 0}) deliberately keep in the database.
For each such row it does attachment.write({"datas": attachment.datas}). The write uploads the content to the object storage, after which the datas inverse re-applies the force-db rules and stores the content straight back into db_datas. The result of each run:
- Zero net effect for those rows: the before/after distribution is identical (verified: same counts on storage and in the database,
store_fname and fs_storage_id agreeing on every row).
- One orphaned object per row per run: the upload is never cleaned up when the row is re-homed to the database. The bucket accumulates unreferenced objects on every invocation.
- Runtime inflated by minutes of pointless upload work, which matters when
force_storage() runs inside a maintenance window or a deployment pipeline.
Observed in production
On an instance with ~1300 database-resident attachments and 147 legitimately storage-resident ones, each force_storage() run took about six minutes, changed nothing, and left ~1300 new orphans. A bucket audit after several routine runs found 739 orphaned objects (~23 MB) against 147 referenced ones; every reference resolved, nothing was broken, but the garbage grows on every run.
A related side effect of the same walk: the write({"datas": ...}) round trip passes through the mimetype re-guess reported in #658, which corrupted text/css asset mimetypes and took a production site's styling down. The two issues are independent (fixing one does not fix the other) but share this code path as the trigger.
To Reproduce
- Configure an fs.storage backend as default for attachments, with the default
force_db_for_default_attachment_rules.
- Use any database with normal content (asset bundles and small images exist in
db_datas).
- Note the object count in the backend and the attachment distribution (
store_fname NULL vs scheme-prefixed).
- Run
env["ir.attachment"].force_storage() twice.
- Observe: distribution unchanged, runtime proportional to the number of database-resident rows on both runs, and the backend object count grown by roughly that number per run.
Expected behavior
force_storage() should not touch rows whose storage decision will put them straight back where they already are. Database-resident attachments that match the force-db rules are already in their configured location; walking them is pure churn.
Suggested fix
Exclude them at selection time: subtract _store_in_db_instead_of_object_storage_domain() from the search domain in _force_storage_to_object_storage() (the domain already exists and is documented as mirroring the decision logic). Alternatively, short-circuit per row before uploading when _store_in_db_instead_of_object_storage(data, mimetype) is true. Either variant makes repeated force_storage() runs idempotent in both effect and cost, and stops the orphan growth.
Module: fs_attachment
Branches affected: verified on 19.0 (19.0.1.0.0) against the pinned source; the mechanism (
_force_storage_to_object_storage+ the force-db rules re-applied by thedatasinverse) is present on 16.0 through 18.0 as well.Describe the bug
ir.attachment.force_storage()selects every attachment whosestore_fnameis not on the object storage. That domain includes database-resident rows (store_fnameNULL, content indb_datas), which on a stock database number in the hundreds to thousands: asset bundles and small images that the defaultforce_db_for_default_attachment_rules({"image/": 51200, "application/javascript": 0, "text/css": 0}) deliberately keep in the database.For each such row it does
attachment.write({"datas": attachment.datas}). The write uploads the content to the object storage, after which thedatasinverse re-applies the force-db rules and stores the content straight back intodb_datas. The result of each run:store_fnameandfs_storage_idagreeing on every row).force_storage()runs inside a maintenance window or a deployment pipeline.Observed in production
On an instance with ~1300 database-resident attachments and 147 legitimately storage-resident ones, each
force_storage()run took about six minutes, changed nothing, and left ~1300 new orphans. A bucket audit after several routine runs found 739 orphaned objects (~23 MB) against 147 referenced ones; every reference resolved, nothing was broken, but the garbage grows on every run.A related side effect of the same walk: the
write({"datas": ...})round trip passes through the mimetype re-guess reported in #658, which corruptedtext/cssasset mimetypes and took a production site's styling down. The two issues are independent (fixing one does not fix the other) but share this code path as the trigger.To Reproduce
force_db_for_default_attachment_rules.db_datas).store_fnameNULL vs scheme-prefixed).env["ir.attachment"].force_storage()twice.Expected behavior
force_storage()should not touch rows whose storage decision will put them straight back where they already are. Database-resident attachments that match the force-db rules are already in their configured location; walking them is pure churn.Suggested fix
Exclude them at selection time: subtract
_store_in_db_instead_of_object_storage_domain()from the search domain in_force_storage_to_object_storage()(the domain already exists and is documented as mirroring the decision logic). Alternatively, short-circuit per row before uploading when_store_in_db_instead_of_object_storage(data, mimetype)is true. Either variant makes repeatedforce_storage()runs idempotent in both effect and cost, and stops the orphan growth.