Contents Menu Expand Light mode Dark mode Auto light/dark mode
Openverse documentation
Light Logo Dark Logo
  • Django API
    • Guides
      • Quickstart guide
      • Testing guide
      • Deployment runbook
      • API Documentation Guidelines
      • Local HTTPS testing
    • Reference
      • Made with Openverse
      • Search Algorithm
      • Authentication and Throttling
      • Django Healthcheck
  • Automations
    • Guides
      • Quickstart Guide
    • Reference
      • Python Scripts
      • Project Automations
  • Airflow Catalog
    • Guides
      • Quickstart
      • Deployment runbook
      • Openverse Providers
      • Catalog Deployments
      • Airflow
      • Postgres
    • Reference
      • DAGs
      • Retired
  • Changelogs
    • API changelogs
      • 2024.04.02.05.06.52
      • 2024.04.01.17.07.10
      • 2024.03.25.15.22.26
      • 2024.03.18.15.51.25
      • 2024.03.13.17.10.22
      • 2024.03.11.15.26.16
      • 2024.03.04.15.15.13
      • 2024.02.28.16.32.28
      • 2024.02.20.22.54.08
      • 2024.02.13.05.45.50
      • 2024.02.05.20.34.01
      • 2024.01.29.19.36.26
      • 2024.01.16.17.28.14
      • 2024.01.08.20.06.53
      • 2024.01.01.19.52.14
      • 2023.12.26.05.11.22
      • 2023.12.18.17.56.24
      • 2023.12.11.19.20.45
      • 2023.12.04.19.51.23
      • 2023.11.29.22.20.31
      • 2023.11.27.05.10.48
      • 2023.11.20.09.32.25
      • 2023.11.14.23.44.01
      • 2023.11.13.22.18.21
      • 2023.11.01.21.27.09
      • 2023.10.30.16.26.21
      • 2023.10.23.11.40.59
      • 2023.10.18.15.25.27
      • 2023.10.11.04.49.20
      • 2023.10.10.21.08.33
      • 2023.10.10.18.42.39
      • 2023.10.06.17.13.36
      • 2023.09.30.00.15.32
      • 2023.09.28.00.26.34
      • 2023.09.25.21.34.44
      • 2023.09.20.11.16.51
      • 2023.09.20.07.37.52
      • 2023.09.19.08.13.24
      • 2023.09.15.19.15.58
      • 2023.09.11.16.56.52
      • 2023.09.04.19.52.19
      • 2023.08.28.20.43.01
      • 2023.08.22.04.57.04
      • 2023.08.14.16.35.15
      • 2023.08.08.22.22.02
      • 2023.07.24.20.47.04
      • 2023.07.17.21.05.23
      • 2023.07.10.08.44.19
      • 2023.07.03.17.52.00
      • 2023.06.28.19.49.58
      • 2023.06.28.02.17.04
      • 2023.06.26.20.33.46
      • 2023.06.19.17.48.39
      • 2023.06.13.00.21.51
      • 2023.06.05.15.04.02
      • 2023.06.01.01.50.43
      • 2023.05.30.17.57.04
      • 2023.05.30.12.02.04
      • 2023.05.19.22.08.07
      • 2023.05.19.18.58.25
      • 2023.05.19.17.16.50
      • 2023.05.15.21.51.49
      • 2023.05.02.21.26.28
      • 2023.04.27.07.29.23
      • 2023.04.23.23.22.14
      • 2023.04.19.00.01.39
      • 2023.04.18.15.27.15
      • 2023.04.12.23.29.59
    • Catalog changelogs
      • 2024.03.22.17.45.11
      • 2024.03.06.16.40.36
      • 2024.01.25.17.42.59
      • 2024.01.04.20.39.50
      • 2023.12.20.22.19.13
      • 2023.11.29.15.35.59
      • 2023.11.14.18.24.16
      • 2023.10.06.18.29.33
      • 2023.08.30.17.30.22
      • 2023.07.20.15.07.11
      • 2023.06.19.17.21.41
      • 2023.06.08.16.52.36
      • 2023.05.16.22.14.49
      • 2023.04.27.02.43.13
    • Frontend changelogs
      • 2024.04.01.17.07.11
      • 2024.03.25.15.22.24
      • 2024.03.18.15.51.41
      • 2024.03.11.15.25.54
      • 2024.03.04.15.15.07
      • 2024.02.26.18.58.35
      • 2024.02.19.15.45.00
      • 2024.02.12.23.34.43
      • 2024.02.05.18.10.41
      • 2024.01.29.19.41.17
      • 2024.01.08.19.12.18
      • 2024.01.01.19.52.49
      • 2023.12.11.19.44.28
      • 2023.12.04.19.35.53
      • 2023.11.27.19.20.44
      • 2023.11.20.09.09.57
      • 2023.11.14.19.30.29
      • 2023.11.06.20.32.30
      • 2023.11.01.00.18.44
      • 2023.10.30.16.27.33
      • 2023.10.09.17.18.13
      • 2023.09.25.22.11.48
      • 2023.09.18.10.25.25
      • 2023.09.11.17.34.23
      • 2023.09.05.17.09.13
      • 2023.09.04.20.20.34
      • 2023.08.28.19.38.22
      • 2023.08.22.06.04.39
      • 2023.08.14.16.26.44
      • 2023.08.08.21.19.44
      • 2023.07.28.23.42.26
      • 2023.07.24.21.23.45
      • 2023.07.20.17.25.42
      • 2023.07.19.19.15.39
      • 2023.07.17.21.49.20
      • 2023.07.10.08.29.25
      • 2023.07.06.14.52.49
      • 2023.07.05.17.56.39
      • 2023.07.03.18.19.32
      • 2023.06.26.20.22.48
      • 2023.06.19.17.31.24
      • 2023.06.13.00.28.40
      • 2023.06.05.14.48.06
      • 2023.05.23.17.02.26
      • 2023.05.22.15.55.57
      • 2023.05.15.22.13.27
      • 2023.05.02.19.17.32
      • 2023.04.23.23.07.51
    • Ingestion server changelogs
      • 2024.03.06.16.40.36
      • 2024.02.28.18.32.08
      • 2024.02.07.18.41.16
      • 2024.01.18.18.31.08
      • 2023.12.12.17.23.14
      • 2023.12.06.19.13.35
      • 2023.11.14.18.39.29
      • 2023.10.25.06.38.07
      • 2023.10.19.06.08.15
      • 2023.10.06.18.02.20
      • 2023.09.29.17.40.50
      • 2023.09.05.18.56.18
      • 2023.08.29.17.36.35
      • 2023.07.21.20.19.21
      • 2023.07.05.17.46.34
      • 2023.06.13.00.25.04
      • 2023.06.07.17.56.30
      • 2023.05.17.22.57.37
      • 2023.05.03.22.29.52
  • Nuxt frontend
    • Guides
      • Quickstart guide
      • Deployment runbook
      • Running frontend tests
      • Analytics
      • SVG Icon Sprites in Openverse Frontend
      • Translate
    • Reference
      • Internationalization
      • Feature flags
      • Testing Guidelines
      • Playwright tests
      • Storybook tests
      • Miscellaneous notes on the frontend
      • Nuxt Healthcheck
      • Frontend reverse proxy
  • General development guidelines
    • Welcome
    • General setup guide
    • Quickstart guide
    • Run
    • Test
    • Zero Downtime and Database Management
    • Deployments
    • Logging
    • Testing HTTPS
    • Stack
  • Ingestion server
    • Guides
      • Quickstart guide
      • Configuration
      • Mapping database tables to Elasticsearch
      • Testing guide
      • Deployment runbook
      • Index migration runbook
      • Manual index upgrade runbook
      • Troubleshooting
    • Reference
      • Elasticsearch and Indexing
      • Safety and Security Considerations
      • Notifications
      • Data Refresh Limit
      • Ingestion server API
  • Managing the Openverse
    • Responding to Incidents
      • Runbooks
        • Incident Response Runbook
    • Contributing to Openverse
      • GitHub contribution practices
      • Development workflow
      • Current maintainers
      • Communication aliases
      • Becoming a Committer
      • Regular maintainer tasks
      • Codespell Configuration : spellcheck pre-commit hook
    • Project boards
      • Issues project: Openverse Backlog
      • PR project: Openverse PRs
      • Project project: Openverse Project Tracker
      • Discussion project: Openverse Discussions
    • CI + CD workflow
      • Actions
      • Jobs
        • Preparation jobs
        • Frontend jobs
        • Packages jobs
        • Docker jobs
        • Catalog jobs
        • Ingestion server jobs
        • API jobs
        • Documentation jobs
        • Deployment jobs
        • Notification jobs
      • Flow
      • Proof-of-functionality
      • Artifacts
    • Openverse Decision-Making Process
      • Process description
      • How to follow the process in different settings
      • Additional practices
    • Documentation
      • Quickstart guide
      • Documentation guidelines
      • Sphinx extensions
      • Documentation proofreading and review
    • Monitoring the Openverse
      • Traffic Management
        • Identifying and Blocking Traffic Anomalies in Cloudflare
      • Log querying
      • Runbooks
        • Run Book: API Production HTTP 2XX responses count under threshold
        • Run Book: API Production HTTP 5XX responses count above threshold
        • Run Book: API Production Average Response Time above threshold
        • Run Book: API Production Average Response Time anomaly
        • Run Book: API Production P99 Response Time above threshold
        • Run Book: API Production P99 Response Time anomaly
        • Run Book: API Production Request Count anomalously high
        • Run Book: Nuxt 2XX responses count under threshold
        • Run Book: Nuxt 5XX responses count above threshold
        • Run Book: Nuxt Production Average Response Time above threshold
        • Run Book: Nuxt Production Average Response Time anomalously high
        • Run Book: Nuxt Production P99 Response Time above threshold
        • Run Book: Nuxt Production P99 Response Time anomalously high
        • Run Book: Nuxt Request Count anomalously high
        • Run Book: Unhealthy hosts for ECS service
    • Openverse Infrastructure Maintenance
      • Elasticsearch cluster maintenance
  • Node.js packages
    • @openverse/eslint-plugin
      • analytics-configuration
      • no-unexplained-disabled-test
  • Projects
    • Project Planning
    • Template: Implementation Plan
    • Template: Project Directory index.md
    • Template: Project Proposal
    • Yearly Planning
      • Yearly Planning: Process Outline
      • Project Themes
    • 3D Model Support
      • 2022-02-21 Project Proposal: 3D model support
    • Additional Search Views
      • 2023-04-24 Project Proposal: Additional search views
      • 2023-07-20 Implementation Plan: Additional Search Views
      • SUPERSEDED 2023-07-20 Implementation Plan: Additional Search Views
    • Analytics
      • 2022-10-06 Implementation Plan: Frontend event tracking
      • 2023-03-07 Implementation Plan: Analytics backend and visualisation service
    • Core User Interface Improvements
      • 2023-03-14 Project Proposal: Core UI Improvement
    • Dark Mode
      • 2024-03-13 Project Proposal
    • Data Normalization
      • 2024-02-27 Implementation Plan: Catalog Data Cleaning
    • Document all media properties
      • 2023-03-07 Project proposal for Document all media properties
      • 2023-04-26 Implementation Plan: Document all media properties in the catalog
      • 2024-03-25 Implementation Plan: Document all media properties in the API
      • Samples
        • Openverse Media Properties
        • sample_media_properties.md
        • sample_postamble.md
        • sample_preamble.md
    • Samples
      • Openverse Media Properties
      • sample_media_properties.md
      • sample_postamble.md
      • sample_preamble.md
    • Feature Flags
      • 2022-03-09 Project Proposal: Feature flags
    • Frontend UI State Cookie
      • 2022-02-18 Project Propsal: Frontend UI State Cookie
    • Openverse Infrastructure
      • 2024-01-05 Implementation Plan: Finish moving all services to openverse.org
    • Ingestion server removal
      • 2024-03-19 Project Proposal: Removal of the Ingestion Server
    • Monitoring
      • 2022-03-07 Project Proposal: Monitoring
      • 2023-06-06 Implementation Plan: Baseline Alarms for ECS Services
        • Stable Run Book Sample
        • Unstable Run Book Sample
    • Monorepo
      • 2022-11-24 Project Proposal: Monorepo
    • Nuxt 3 Migration
      • 2023-06-02 Project Proposal
    • Popularity Calculation Optimizations
      • 2023-04-06 Project Proposal: Popularity Calculation Optimizations
      • Implementation Plan: Decoupling Popularity Calculations from the Data Refresh
    • Project board improvements
      • 2023-09-13 Project proposal: Project board improvements
      • 2023-09-14 Implementation plan: Project automations
    • Providing and maintaining an Openverse image dataset
      • 2023-07-06 Project Proposal: Providing and maintaining an Openverse image dataset
    • Search Relevancy Sandbox
      • 2023-03-31 Project Proposal: Search relevancy sandbox
      • 2023-04-06 Implementation Plan: Update Staging Database
      • 2023-05-18 Implementation Plan: Rapid iteration of ingestion server index configuration
      • 2023-06-08 Implementation Plan: Staging Elasticsearch Reindex DAGs
    • Sensitive Content Report Moderation Initial Practices
      • 2023-04-11 Project Proposal: Sensitive content report moderation: initial practices
      • 2023-05-17 Implementation Plan: Copy updates mature -> sensitive
      • 2023-12-08 Implementation Plan: Django admin tools and access control for moderators
      • 2024-01-22 Implementation Plan: Bulk Moderation Actions
      • 2024-01-22 Implementation Plan: Moderation response times
    • Detecting and Blurring Sensitive Textual Content
      • 2023-03-09 Implementation Plan: Sensitive Terms List
      • 2023-03-09 Project Proposal: Detecting sensitive textual content and blurring all sensitive results
      • 2023-03-30 Implementation Plan: Filtering and designating results with sensitive textual content
      • 2023-05-06 Implementation Plan: Fetching, blurring sensitive results
    • Trust and Safety
      • 2023-01-09 Initial Overview and User Stories
      • Sensitive Content Report Moderation Initial Practices
        • 2023-04-11 Project Proposal: Sensitive content report moderation: initial practices
        • 2023-05-17 Implementation Plan: Copy updates mature -> sensitive
        • 2023-12-08 Implementation Plan: Django admin tools and access control for moderators
        • 2024-01-22 Implementation Plan: Bulk Moderation Actions
        • 2024-01-22 Implementation Plan: Moderation response times
      • Detecting and Blurring Sensitive Textual Content
        • 2023-03-09 Implementation Plan: Sensitive Terms List
        • 2023-03-09 Project Proposal: Detecting sensitive textual content and blurring all sensitive results
        • 2023-03-30 Implementation Plan: Filtering and designating results with sensitive textual content
        • 2023-05-06 Implementation Plan: Fetching, blurring sensitive results
    • Visual Regression Testing
      • 2022-02-17 Implementation Plan: Visual regression testing
    • Pinia Conversion
      • 2022-02-23 Project Proposal: Vuex to Pinia conversion
  • Openverse Terms of Service

External links

  • Openverse
  • GitHub
Back to top
Edit this page

Providing and maintaining an Openverse image dataset#

  • 2023-07-06 Project Proposal: Providing and maintaining an Openverse image dataset
Next
2023-07-06 Project Proposal: Providing and maintaining an Openverse image dataset
Previous
2023-09-14 Implementation plan: Project automations
Made with Sphinx and @pradyunsg's Furo