InSpect Dataset

InSpect Dataset

On this page

InSpect Overview

InSpect overview with three panels: (a) collection images and labels; (b) aligned specimen crops, hierarchical taxonomy, structured metadata, and spatial-aware and singular part masks; (c) open taxonomic recognition and anatomical part segmentation benchmarks.
Figure 2: Overview of InSpect and its two benchmarks. (a) The dataset contains 48,184 natural-history collection images spanning 653 taxa, with specimens and their accompanying collection labels. (b) Each specimen is associated with an aligned crop, hierarchical taxonomy, and label-derived structured metadata. A subset of 1,859 images provides fine-grained anatomical part masks in two forms: ① spatial-aware and ② singular. (c) InSpect supports open taxonomic recognition from specimen crops with optional metadata under zero-shot, fine-tuned, and unseen-taxon settings, as well as anatomical part segmentation under supervised and text-guided settings.

Data Structure

The release is hosted on Hugging Face. Images and anatomical annotations are distributed as ZIP archives; specimen metadata is provided in CSV and JSONL formats.

InSpect/
├── README.md
├── original_images.zip
├── final_benchmark_data_crops.zip
├── specimen_benchmark_metadata.csv
├── specimen_benchmark_metadata.jsonl
├── segmentation_images.zip
└── segmentation_annotations.zip
FileSizeContents
original_images.zip248.15 GBOriginal collection photographs, including specimens and their physical labels.
final_benchmark_data_crops.zip17.50 GBCropped insect images for visual recognition; use these to exclude physical label text from the visual input.
specimen_benchmark_metadata.csv32.07 MBOne row per specimen; image identifiers, taxonomy, benchmark splits and label-derived context.
specimen_benchmark_metadata.jsonl57.80 MBJSONL version of the specimen metadata.
segmentation_images.zip801.91 MBImages for the anatomical part segmentation subset.
segmentation_annotations.zip13.96 MBCOCO-style JSON annotations for singular and spatial-aware part labels.

File sizes checked on 30 October 2026; decimal units (1 GB = 1,000,000,000 bytes). Download only the files needed for your task.

Recognition: start with final_benchmark_data_crops.zip and either metadata file. Part segmentation: use segmentation_images.zip with segmentation_annotations.zip. Original full images are available separately for studying collection images and labels.

Metadata fields and annotation files

The released CSV contains 48,184 records and 31 columns. Key identifiers are image_name and catalog_number; taxonomy is stored in order, family, genus, species and taxon_key. The split column uses train, val, test_seen and test_unseen. Context fields include locality, collection date, collector, life stage and sex, with missing values preserved.

For metadata-based recognition, exclude direct taxonomic fields as described in the paper. The annotation archive contains the following files (filenames, including spaces, are shown as released):

annotations/
├── singular_sam3_train.json
├── singular_sam3_val.json
├── singular_sam3_test.json
├── sam3_train 1.json
├── sam3_val 1.json
└── sam3_test 1.json

Each JSON file contains COCO-style images, annotations and categories entries. The singular files contain 11 singular anatomical labels; the other three files use a 31-category vocabulary with spatial-aware distinctions.

Annotations Examples

Recognition

Explore a specimen’s original collection photograph, aligned insect crop, hierarchical taxonomy and label-derived metadata. Select a specimen below to inspect the corresponding record.

Click the image to enlarge
Braunsapis protuberansSpecimen 32-065827 · test_unseen
Original collection image of Braunsapis protuberans; specimen 32-065827Aligned insect crop of Braunsapis protuberans; specimen 32-065827

The original photograph includes physical labels. The crop isolates the specimen for visual recognition.

Biological taxonomy

Order
Hymenoptera
Family
Apoidea
Genus
Braunsapis
Species
protuberans

Structured metadata

Locality
5Km WbyN Rounded Hill nr Hope Vale Mission Q
Collection date
7 Oct 1980
Collector
J.C. Cardale
Sex
Not recorded
Collection
ANIC
View full specimen record

All fields below are reproduced from the released metadata, including the supplied label_text and weak_context_text fields.

Download this record (JSON)

image_name
32-065827.jpg
catalog_number
32-065827
scientific_name
Braunsapis protuberans
order
Hymenoptera
family
Apoidea
genus
Braunsapis
species
protuberans
authority
Reyes
year
Not recorded
locality
5Km WbyN Rounded Hill nr Hope Vale Mission Q
original_locality
Not recorded
state_province
Queensland
verbatim_coordinates
15.17S 145.10E
latitude
-15.17
longitude
145.1
collection_date
7 Oct 1980
collector
J.C. Cardale
sampling_event
Not recorded
life_stage
Not recorded
sex
Not recorded
determiner
Not recorded
determination_date
Not recorded
institution
ANIC
notes
PARATYPE; on flowers Melaleuca foliolosa
notes_clean
PARATYPE; on flowers Melaleuca foliolosa
weak_context_text
Locality: 5Km WbyN Rounded Hill nr Hope Vale Mission Q. Original locality: unknown. State or province: Queensland. Latitude: -15.17. Longitude: 145.1. Collection date: 7 Oct 1980. Collector: J.C. Cardale. Sampling event: unknown. Life stage: unknown. Sex: unknown. Determiner: unknown. Determination date: unknown. Notes: PARATYPE; on flowers Melaleuca foliolosa.
label_text
Order: Hymenoptera. Family: Apoidea. Genus: Braunsapis. Species: protuberans.
taxon_key
Braunsapis__protuberans
split
test_unseen
species_split
unseen

Source: InSpect release. Original photographs, recognition crops and metadata are matched by image filename. Taxonomy entries are dataset labels; metadata values are preserved as released. Missing fields are shown as “Not recorded”. Photographs are resized for web display.

Open image for closer inspection ↗

Segmentation

Explore anatomical part annotations from InSpect. Select a specimen, switch between singular and spatial-aware labels, or adjust the mask opacity to inspect the underlying image. These overlays show released ground-truth annotations, not model predictions.

Braunsapis protuberans, specimen 32-065827, with annotated anatomical parts.
Labels
  • abdomen
  • cercus
  • head
  • leg
  • thorax
  • tongue
  • wing
  • abdomen
  • head
  • left cercus
  • left forewing
  • left hindleg
  • left hindwing
  • left middleleg
  • right foreleg
  • right forewing
  • right hindleg
  • right middleleg
  • thorax
  • tongue
Braunsapis protuberans · Hymenoptera
Specimen 32-065827 · 14 annotated parts · Singular14 annotated parts · spatial-aware

Images and masks: InSpect release, segmentation_images.zip and segmentation_annotations.zip. Colors identify labels consistently across specimens within each annotation mode. Only labels present in the selected specimen are shown.

Tools

Code: Recognition and Segmentation

Contribute

We welcome contributions of the following:

  • Published benchmarking results of algorithms applied to the InSpect dataset.
  • Corrections to existing descriptions or ground truth.
  • Other aspects of the InSpect.

Please email junhaozhang.ai@gmail.com if you have any questions about contributing datasets.

Citation

If you find our InSpect dataset is useful in your research, please consider cite:

Provisional BibTeX; replace with the official export when available.

@misc{zhang2026inspect,
  title = {InSpect: A Curated Natural History Collection Dataset
           for Insect Specimen Understanding},
  author = {Zhang, Junhao and Xu, Feng and Ravendran, Ahalya
            and Fisher, Nicole and Tidd, Brendan
            and Petersson, Lars and Wang, Dadong and Li, Xun},
  year = {2026},
  note = {NeurIPS 2026; provisional metadata},
  url = {https://openreview.net/forum?id=51J0UrBZZy}
}

  Download .bib

Copyright:

Our dataset follows the copyright Creative Commons BY-NC-ND 4.0 license.

Search