Open Metadata Exchange plugin for QUBES

An OME plugin defines how to import and publish metadata from an Open Education Resources system.

plugin.py provides the class that enables OME to understand QUBES metadata.

This plugin defines one or more:

  • mimetypes so that QUBES json data can be stored as enclosures in InterNetNews (INN) articles.

  • newsgroups so QUBES metadata can be published to and subscribed from those INN newsgroups.

The plugin is composed of at least three code files.

  • plugin.py: Defines mimetypes, newsgroups, and a make_metadata_card() function that translates the Open Education Resources system’s data into an OME EducationalResource.

  • qubes_models.py: generated by Pydantic tools to make it fast and reliable to import QUBES metadata.

  • load_qubes_records_to_nntp.py: Convert qubes_records.json into OME EducationalResources and load them into the default NNTP server.

  • convert_qubes_xml_to_json.py: Data utility that can probably be dropped by adding the dependency pydantic-xml.

server/plugins/qubes/
├── convert_qubes_xml_to_json.py
├── load_qubes_records_to_nntp.py
├── plugin.py
├── qubes_models.py
├── qubes_records.json
└── qubes_records.xml

Steps to import QUBES metadata and post it as InterNetNews articles.

  1. Use cURL or wget to create a qubes_records.xml file as discussed in convert_qubes_xml_to_json.py.

  2. Run server/plugins/qubes/convert_qubes_xml_to_json.py to generate qubes_records.json.

  3. [Optional]: Run server/plugins/qubes/qubes_models.py to ensure that qubes_records.json can be parsed.

  4. [Optional]: Run PYTHONPATH="." server/plugins/qubes/plugin.py to ensure that qubes_records.json can be parsed.

  5. docker compose build

  6. open http://localhost:5001/newsgroups && docker compose up – The webpage will be blank until the FastAPI server starts.

  7. PYTHONPATH="." scripts/create_newsgroups.py

  8. Refresh webpage to show that the newsgroups were created.

  9. [Optional]: open http://localhost:5001/api/channel/ome.qubes – Ensure total=0, first=1, last=0

  10. server/plugins/qubes/load_qubes_records_to_nntp.py

  11. [Optional]: Refresh the webpage and ensure total=51, first=1, last=51

  12. Change URL from 5001 (Austin) to 5002 (Boston) to ensure total=0, first=1, last=0

  13. PYTHONPATH="." scripts/nntp_sync.py to transfer Austin articles to Boston…

  14. Refresh web page to ensure total=51, first=1, last=51

  15. open http://localhost:5001/api/channel/ome.qubes/cards to show individual records

QUBES API

Gets list of publications https://qubeshub.ddev.site/oaipmh/?verb=ListRecords&metadataPrefix=qdc&set=publications

Gets metadata for a specific publication based on url https://qubeshub.ddev.site/oaipmh/?verb=GetRecord&metadataPrefix=qdc&identifier=https://qubeshub.ddev.site/publications/1/1

Setting up newsgroups

Currently not working:

PYTHONPATH="." scripts/create_newsgroups.py

Manual method (might need to only run newgroup commands one, even after docker restart).

docker compose up
docker exec -it ome-internetnews-server-austin-1 sh -c "ctlinnd newgroup ome.eric"
docker exec -it ome-internetnews-server-austin-1 sh -c "ctlinnd newgroup ome.oer"
docker exec -it ome-internetnews-server-austin-1 sh -c "ctlinnd newgroup ome.openlibrary"
docker exec -it ome-internetnews-server-austin-1 sh -c "ctlinnd newgroup ome.qubes"
docker exec -it ome-internetnews-server-austin-1 sh -c "ctlinnd newgroup ome.whg"

Visit http://localhost:5001/.