Where to get them: the embedding files are
not stored in this git repository.
They are released as a Zenodo deposit at
10.5281/zenodo.20057840.
After cloning the repo, copy the Zenodo files into the empty
embeddings_output/ directory at the project root and the rest of the project will pick them up.
Note on entity counts (rdflib vs PyKEEN).
All KG-size figures elsewhere on this page (e.g. 656,121 unique RDF nodes,
263,343 literal nodes) are computed by parsing
data/kg/imdb_kg_cleaned.ttl with rdflib. The PyKEEN entity table that
backs the released KG embeddings has 656,003 rows instead — 118 fewer than
rdflib reports — because PyKEEN dedupes literals more aggressively than rdflib (literals
with identical lexical form but different datatypes or language tags are collapsed into one
entity). The 118-node delta is entirely in the literal block (rdflib: 263,343 distinct literals;
PyKEEN: 263,225). Concretely, the released bundle contains 139,465 rows in the
URIRef KG-embedding table (the table most users consume — unaffected by the dedup difference)
and ~656,003 rows in the optional /kg_pykeen_entities/ HDF5 group.
Both numbers describe the same KG; they differ only in how the literal namespace is collapsed.
Released Vectors
| Modality |
Rows |
Dim |
Model |
| Image |
33,247 |
768 |
openai/clip-vit-large-patch14 |
| Video |
4,350 |
512 |
microsoft/xclip-base-patch32 |
| Audio |
4,034 |
512 |
laion/larger_clap_music_and_speech |
| Text |
4,216 |
1,024 |
BAAI/bge-large-en-v1.5 |
| KG (RotatE) |
139,465 |
512 |
PyKEEN RotatE (256-d complex / 512-d real) |
Held-out KG Variants
RotatE is additionally trained with specific label predicates removed before training,
supporting clean classification benchmarks. The all-labels run held out
66,838 triples (1,731,988 used for training).
| Variant |
Held-out Predicate(s) |
Purpose |
full |
none |
KG retrieval, projections, fusion |
genre |
schema:genre |
Clean genre classification |
rating |
schema:contentRating |
Clean rating classification |
decade |
schema:datePublished |
Clean decade classification |
language |
schema:inLanguage |
Clean language classification |
all-labels |
the four above, jointly |
Stricter robustness setting |
Embedding Quality (Leave-one-out NCC accuracy)
How well each modality (and several fusion strategies) recovers KG-derived labels for the
352-movie multimodal intersection. Headline numbers from plots/out/tab_fusion_metrics.tex.
| Modality / Fusion |
Decade (5) |
Rating (4) |
Genre (9) |
| Random baseline |
20.0% |
25.0% |
11.1% |
| Poster (CLIP ViT-L/14) |
77.6% |
61.2% |
37.5% |
| Trailer (X-CLIP) |
44.6% |
56.9% |
30.1% |
| Soundtrack (CLAP) |
34.9% |
29.5% |
13.8% |
| KG (RotatE, label-held-out) |
38.4% |
48.0% |
25.8% |
| Text balanced avg. (BGE) |
40.3% |
64.3% |
50.1% |
| Fused, late, poster-weighted (2:1:1:1:1) |
65.6% |
63.7% |
42.7% |
| Fused, supervised CV late |
80.0% ± 4.2 |
61.6% ± 2.2 |
57.4% ± 6.0 |
Alignment guarantee: every row in the released parquet files has a matching
subject in data/kg/imdb_kg_cleaned.ttl. Each modality is delivered as a Parquet
(zstd) file and an HDF5 (gzip-4) group with shared row order, so
parquet_row == hdf5_index.
