Cyberbiosecurity Literature Tracker

An automated pipeline that monitors PubMed for new research at the intersection of biosecurity, biosafety, and cybersecurity — a fast-growing but still loosely-defined field ("cyberbiosecurity") — and delivers results as a live, queryable dashboard in Obsidian.

Why this exists

"Cyberbiosecurity" is a young term, so relevant papers are scattered across adjacent keywords (biosecurity, DNA synthesis screening, genomic data security, etc.) rather than indexed under one consistent tag. This project searches across that full keyword set on a schedule, so new literature surfaces automatically instead of requiring manual searches across multiple terms.

How it works

GitHub Actions (weekly)
        │
        ▼
cyberbiosecurity_scraper.py
        │
        ├─ PubMed E-utilities API ─────────► peer-reviewed papers
        ├─ Europe PMC API (SRC:PPR) ───────► bioRxiv / medRxiv preprints
        └─ arXiv API ───────────────────────► arXiv preprints (cs.CR etc.)
        │
        ▼
 ┌───────────────┬────────────────────┐
 │ CSV backup     │ Markdown notes     │
 │ (spreadsheet)  │ (one per result,   │
 │                │  with DOI + tags)  │
 └───────────────┴────────────────────┘
        │
        ▼
   Auto-committed to this repo
        │
        ▼
  Pulled into Obsidian (via Obsidian Git)
        │
        ▼
  Dataview queries render live dashboard tables

Preprints

Peer-reviewed publication can lag 6–12+ months behind when research is actually posted, so this project also tracks preprints — often the earliest signal in a fast-moving field like this one:

• bioRxiv / medRxiv — pulled via the Europe PMC API (SRC:PPR filter), since neither preprint server has reliable keyword search on its own
• arXiv — pulled via arXiv's own API, useful for the cybersecurity/AI side of this topic (e.g. the cs.CR category)

Every result — peer-reviewed or preprint — gets a status field in its frontmatter (peer-reviewed or preprint), so the dashboard can filter or group by publication stage. Example query for a preprints-only table:

```dataview
table authors, journal, year, doi
from "obsidian_notes"
where status = "preprint"
sort year desc
```

Tech stack

• Python (requests, xml.etree.ElementTree) — scraping and parsing
• PubMed E-utilities API — peer-reviewed literature
• Europe PMC API — bioRxiv/medRxiv preprint search
• arXiv API — cybersecurity/AI-adjacent preprints
• GitHub Actions — free scheduled automation, no server required
• Obsidian + Dataview + Charts + Obsidian Git — dashboard, visualizations, and vault sync

Setup

1. Clone this repo.
2. Ensure .github/workflows/scrape.yml is present — this defines the schedule.
3. In repo Settings → Actions → General → Workflow permissions, select "Read and write permissions" so the workflow can commit new notes.
4. (Optional, local run) Install dependencies and run manually:

   pip install requests
   python cyberbiosecurity_scraper.py
   

5. For a ready-made dashboard instead of building queries from scratch, see Dashboard template below.

Dashboard template

dashboard-template.md in this repo is a drop-in Obsidian dashboard: a live search box (by title, author, or journal), a theme radar chart comparing peer-reviewed vs. preprint coverage across five research themes (AI safety, cyber threats, bio threats, governance, surveillance), two horizontal bar charts (papers by year, and preprint vs. published), plus Dataview tables for recently added papers, peer-reviewed vs. preprint listings, a year-by-year grouped list, and quick counts.

To use it:

1. Install the Dataview and Charts community plugins in your vault.
2. Clone this repo into your vault (or wherever Obsidian Git can reach it).
3. Copy dashboard-template.md into your vault — outside the cloned repo folder, so it stays a personal note rather than getting committed back here — and open it in Obsidian.
4. Update the folder path in each query to match wherever you cloned this repo inside your own vault.

This file is a template only — it contains no paper data itself, just the query/chart logic. It won't render as a working dashboard when viewed on GitHub (GitHub doesn't execute Dataview/Charts code), only inside Obsidian with those plugins installed.

Configuration

• Keywords: edit the KEYWORDS list (PubMed) and PLAIN_KEYWORDS list (Europe PMC/arXiv) at the top of cyberbiosecurity_scraper.py.
• Schedule: edit the cron line in .github/workflows/scrape.yml (crontab.guru is handy for building the expression).
• Result count per run: RESULTS_PER_KEYWORD (PubMed) and PREPRINT_RESULTS_PER_KEYWORD (Europe PMC/arXiv).
• Excluding specific papers: add its ID to excluded_ids.txt (one per line). Find the ID in the note's pmid: frontmatter field. This both stops it from being re-added on future runs and deletes its existing note.

Output

• cyberbiosecurity_papers.csv — flat backup of every paper found, with DOI links and full abstract text
• obsidian_notes/ — one Markdown file per paper (filename = year + title slug + PubMed/preprint ID), with YAML frontmatter (title, display_title, authors, authors_short, year, pub_date, journal, DOI, pmid, status, date_added, themes, tags) for use with Dataview, plus the full abstract in the note body. The themes block scores the paper 0–3 across five research areas (AI safety, cyber threats, bio threats, governance, surveillance) based on which keyword(s) matched — used by the dashboard template's theme radar chart.

Possible extensions

• AI-generated 2–3 sentence summaries in place of full abstracts (small ongoing API cost — currently skipped to keep this project free to run)
• Deduplicate/flag papers that match multiple keyword variants
• Auto-generate a weekly digest email
• Publish the dashboard as a static site (GitHub Pages)

License

MIT — feel free to adapt for your own literature-tracking needs.