Welcome to JacSON: a Javascript and JSON-based scraper and integration tool from UQ Jac Course Profiles into Blackboard Ultra.
This guide explains everything that must be edited if you're adapting JacSON for another school, faculty, or institution.

• View code on Github: https://github.com/uqgblaze/jacson
• Live version: https://uqgblaze.github.io/jacson/

JacSON automates the retrieval of learning outcomes, assessment details, and weekly learning activities from the current (non-archived) version of the UQ course profiles website. These outputs can then be used to pre-populate components in Blackboard Ultra using JavaScript, enabling a more efficient course build process for teaching teams and learning designers.

• 🧠 Course Intelligence: Parses and extracts structured data such as:
  • Learning Outcomes
  • Assessment Tasks (titles, weights, dates, descriptions)
  • Weekly Activities (topics, outcomes, periods)
• ⛔ Smart Filtering: Ignores any links to archived course profiles.
• 📝 Google Sheets Enabled: Reads course codes from a central Google Sheet and updates scrape status and notes for each course
• 🪵 Creates logs and buffers: Uses a local course-list.csv file to determine which courses to scrape or ignore.
• 🗃️ Organized Output: Saves JSON output in folders by semester code.
• ☁️ Automatic Cloud Publishing: Configured to automatically upload JSON results to GitHub repository
• 🧪 Python-based and extensible: Designed to be modular and easy to integrate with existing Python or web workflows.
• ✅ Deploy and copy HTML to any course: You can then copy and deploy the HTML to any course in Blackboard Ultra. It will automatically find the course code and find the scraped JSON file within a document.

• This tool is purpose-built for the UQ JAC system and may not generalize to other institutions without significant modification.
• Course profiles need to be written in a standardised format to benefit from this tool.
• The JSON file can only be called within a document on Blackboard. This script does not work at the course-view level.

Here’s how to replicate JacSON from scratch in your school or faculty.

If you haven't already, download the Repo as a ZIP file. Instructions below assume you've saved it to: %USERPROFILE%\Documents\JacSON

Make sure Python 3.10 or later is installed.

• Download Python
• During install: ✅ Add Python to PATH

Once that is installed, open a command prompt window (CMD.exe, not PowerShell!) and change to the folder: e.g. cd %USERPROFILE%\Documents\JacSON

Then install dependencies:

pip install -r requirements.txt

• If you get an error about 'pip', that means you need to manually install PIP. Google is your friend!

• See link for template: https://docs.google.com/spreadsheets/d/1y4JfTa76oVyWndfA83bAqeN4XwjbMKyWgOyR60yEzXs/edit?usp=sharing

JacSON expects this layout:

• Column B: TRUE to auto-run
• Column C: Status (e.g., Scheduled, Success!, Failure)
• Column D: If TRUE, the course is ignored

JacSON reads and writes to a shared Google Sheet. To enable this:

• Visit: https://console.cloud.google.com/
• Create a new project and enable the Google Sheets API
• Generate a Service Account with a JSON key file
• Share your Sheet with the service account email (e.g., jacson-api@your-project.iam.gserviceaccount.com)

Place your credentials JSON here:

./secrets/credentials.json

• Currently unsupported feature: For GitHub Actions, set as an environment variable GOOGLE_SERVICE_ACCOUNT_JSON.

• A GitHub account
• Access to the target GitHub repository (owner or write access)

Go to: https://github.com/login

• Click your profile icon in the top-right
• Select "Settings"
• Scroll down the left sidebar to "Developer settings"
• Click on "Personal access tokens"
• Then click "Tokens (classic)"

• Click "Generate new token" → "Generate new token (classic)"

Fill out the form:

• Note: Something like JacSON Uploader
• Expiration: Choose a long-lived token if you're automating
• Scopes: ✅ Tick only:
  • repo → full control of private repositories

This includes:

• repo:status
• repo_deployment
• public_repo
• repo:invite

• Click the green "Generate token" button
• 📋 Copy and Save the token immediately — GitHub will not show it again!

1. Open your JacSON folder.

2. Create a file:

   ./secrets/github_token.txt

3. Paste the token into the file:

   ghp_XXXXXXXXXXXXXXXXXXXXXXXXXXXX

✔️ Now JacSON’s upload_profiles.py will authenticate and upload .json

Below is a complete list of files, variables, and settings that you will need to modify to get JacSON running in your own environment.

File: scripts/sheets_updater.py
Lines:

SPREADSHEET_ID = '1tJ04EY1AtyS-7iKlmgZhmom0f97xK3DsZ88wkqmRwNs'
RANGE_NAME     = 'Sheet1!A2:F'
CREDENTIALS_FILE = os.path.join('secrets', 'credentials.json')

• 🔄 Replace SPREADSHEET_ID with your own Google Sheet ID (found in the URL).
• 🔄 Update RANGE_NAME if your data starts in a different cell.
• 📁 Ensure credentials.json exists at ./secrets/credentials.json.

Files:

• scripts/sheets_updater.py
• scripts/update_status.py
• scripts/generate_course_list.py

These scripts expect a service account credential file saved to:

./secrets/credentials.json

File: scripts/upload_profiles.py
Lines:

GITHUB_OWNER = "uqgblaze"
GITHUB_REPO  = "jacson"

• 🔄 Replace with your GitHub username/org and repository name.

File: scripts/upload_profiles.py
Line:

TOKEN_PATH = os.path.join(PROJECT_ROOT, "secrets", "github_token.txt")

• 🔄 Store your GitHub token in this path as a plain text file (no newline).

All Google Sheets integrations assume:

• Column A = Course Code
• Column B = Auto-run checkbox
• Column C = Status (e.g., Scheduled, Success!, Failure)
• Column D = Manual flag (exclusion)
• Column F = Notes

🔄 Adjust in scripts if your Sheet layout differs.

File: jacson.py
Line:

target_domain = "https://course-profiles.uq.edu.au/course-profiles/"

• 🔄 Replace with your institution’s profile system root URL.

File: jacson.py → inside save_course_data()
Line:

profiles_root = os.path.join(base_directory, "profiles")

• 🔄 Change directory if you want to save files somewhere else.

Files: jacson.py, generate_course_list.py, sheets_updater.py

• 🔄 The file course-list.csv is read and written by many scripts. You may rename it, but update all references.

• To test specific courses: edit course-list.csv directly
• To run individual scripts:
  • Pull course list: python scripts/generate_course_list.py
  • Scrape only: python jacson.py
  • Upload only: python scripts/upload_profiles.py
  • Update status:

• To run this automatically from your computer, run either option depending on your operating system: