2023-05-17 Implementation Plan: Copy updates mature -> sensitive#

Author: @aetherunbound

Reviewers#

Overview#

This implementation plan concerns the copy changes necessary for updating references of mature content to sensitive content. These changes will be made to the frontend and Django API, both in code and in web-facing copy. There should be no design changes or new features as part of this plan, only changes to copy/text itself. The initial approach for this effort was to only update field and model names for the API without the need for database migrations (i.e., using Django’s built-in tools for specifying different underlying names for fields and tables). After some discussion however, we decided that now would be an ideal time to perform the migrations since the process will only become more difficult and take longer as the tables/columns grow.

Expected Outcomes#

All references to mature in code and copy will be updated to sensitive, except where mature is part of a data model or makes the most sense to keep (e.g. when interacting with certain provider aspects).

Outlined Steps#

The work is broken up into two services (API and frontend) and those services are broken into two aspects (code and copy).

API#

Code#

The following model names will need to be updated:

The mature reason (and mature_filtered value) in AbstractMediaReport definition will also need to be changed to sensitive and sensitive_filtered respectively. The migration necessary for changing the data is described below, however the API will need to continue to accept mature as a report reason and convert it internally to sensitive (at least until we decide to make a version change to the API). The media report tables already use table aliases as well (ImageReport and AudioReport) - since we’re already performing table renames as part of this implementation plan, it makes sense to rename these tables for the same reason.

In order to perform both of these changes in a zero-downtime manner, the following steps will need to be taken:

  1. Add new models with updated names and fields for SensitiveImage and SensitiveAudio. Also add new models for ImageReport and AudioReport. This will require renaming code references from the existing ImageReport and AudioReport models to NsfwReport and NsfwReportAudio temporarily while maintaining the reference to the existing table name. Said another way:

    • ImageReport -> NsfwReport (code only, db_table should remain the same)

    • AudioReport -> NsfwReportAudio (code only, db_table should remain the same)

    • Add ImageReport (new model, db_table will not need to be overridden)

    • Add AudioReport (new model, db_table will not need to be overridden)

  2. Modify the API to write to both the old and new tables for each of the above instances. On the new report tables, the mature_filtered reason should be written as sensitive_filtered. Data should continue to be read from the original tables during this time.

  3. Deploy this version of the API.

  4. Create a data management command which copies all data from the original tables to the new tables. The identifier field can be used for determining which rows have already been copied.

  5. Deploy this version of the API and run the data management command until no more rows need to be copied for each of the columns.

  6. Modify the ingestion server to use the new tables as reference for determining mature status. Specifically the exists_in_mature table check (this will necessarily involve updating tests and initialization SQL as well).

  7. Remove all code references to the old models (MatureImage, MatureAudio, NsfwReport, NsfwReportAudiofrom the API, but leave the models in place for the time being. Data should now only be written to and read from the new tables.

  8. Deploy this version of the API so that the only versions currently deployed are writing to the new tables.

  9. Remove the old models (MatureImage, MatureAudio, NsfwReport, NsfwReportAudio) from the API codebase and deploy this version of the API. This will remove the old tables from the database.

Per the detecting sensitive textual content project plan, the mature field of the media serializer will remain, and will eventually have a sensitivity field added to it to encapsulate specific information about the reason behind a media being marked sensitive. Those changes are not under the purview of this implementation plan.

The Elasticsearch model property mature will also not be affected as part of this effort.

Copy#

Most changes necessary for updating the copy in the Django Admin UI will be reflected by the code changes that are made above (since the Admin UI introspects the code directly). The only other references that will need to be updated are the help_text values in the following places:

Changes to help_text will produce no-op migrations within Django.

Frontend#

Code#

At present, the following locations within the codebase make reference to the word mature:

  • src/components/VErrorSection/meta/VErrorSection.stories.mdx

  • src/components/VFilters/VFilterChecklist.vue

  • src/components/VFilters/VSearchGridFilter.vue

  • src/constants/content-report.ts

  • src/constants/filters.ts

  • src/stores/media/index.ts

  • src/stores/media/single-result.ts

  • src/stores/search.ts

  • src/types/media.ts

  • src/utils/content-safety.ts

  • src/utils/search-query-transform.ts

  • test/playwright/e2e/report-media.spec.ts

  • test/playwright/e2e/search-query-server.spec.ts

  • test/unit/fixtures/audio.js

  • test/unit/specs/components/v-content-report-form.spec.js

  • test/unit/specs/stores/search-store.spec.js

  • test/unit/specs/utils/search-query-transform.spec.js

The following objects within the codebase will need to be renamed (along with any references that depend on them):

The following references will not be affected because they correspond to either the data model or to the API query parameter mature. The query parameter will be changed to include_sensitive_results in the API as part of another project

Copy#

Almost all copy changes necessary can be made in the en.json5 base translations file. The filters.mature key can be removed entirely, as it references a filter which no longer exists.

Dependencies#

Infrastructure#

This work will include several migrations for the API as well as a deployment for the ingestion server, which will need to be deployed one after another in production as described in the API section.

Tools & packages#

No other tools or packages should be required.

Other projects or work#

This work intersects with, but is distinct from, the “Detecting and Blurring Sensitive Textual Content” project.

Design#

No design work should be necessary.

Parallelizable streams#

The frontend copy work and the API copy & code work can be done simultaneously. The frontend code work should wait until the API work is complete so the content reporting using sensitive rather than mature is available on the API prior to the frontend code changes. It will also need to take into account and coordinate with the changes being made as part of the “Fetching, blurring sensitive results” implementation.

Blockers#

No known blockers at the time of drafting.

API version changes#

If the API backwards compatibility for using mature rather than sensitive on the content reporting endpoint is removed, we will need to change the API versioning scheme. The removal of this backwards compatibility is not strictly a requirement for the success of this IP, so there is not an urgent necessity to change the API version.

Accessibility#

No accessibility changes should be necessary, as this should only affect copy.

Rollback#

In the case that we needed to roll back, the code and copy changes could be easily undone.

The API changes would be more difficult to roll back, as they would require the reverse of the multi-step migrations described in the API section. This would still be feasible, but we should be clear before moving forward that the changes to the table names are ones we wish to make at this juncture.

Localization#

Any changes that we make to the frontend copy will need to have new or existing translations associated with it.

Risks#

We should be careful that any changes made to the API as part of this effort are done in a backwards compatible way, and that we do not introduce any breaking behavior.