Skip to content

#17 - Update list of supported formats #43

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
reckart merged 1 commit into from
Jul 9, 2026
Merged

#17 - Update list of supported formats #43

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 40 additions & 18 deletions docs/api/formats.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,25 +4,47 @@ Documents, annotations and exports can be downloaded/created in different format

INCEpTION doesn't specify in their documentation which formats are supported, but the following have been found and included in `pycaprio`:

* `bin`: Binary.
* `conll2000`: CONLL 2000
* `conll2006`: CONLL 2006
* `conll2009`: CONLL 2009
* `conllcorenlp`: CONLL Core NLP
* `conllu`: CONLLu
* `ctsv`: CTSV
* `ctsv3`: CTSV3
* `dkpro-core-tei`: Dkpro Core TEI
* `html`: HTML
* `lif`: LIF
* `nif`: NIF
* `pdf`: PDF
* `perseus_2.1`: Perseus 2.1
* `pubannotation-sections`: Pubannotation sections
* `tcf`: TCF
* `bin`: UIMA binary CAS
* `bioc`: BioC XML
* `conll2000`: CoNLL 2000
* `conll2002`: CoNLL 2002
* `conll2003`: CoNLL 2003
* `conll2006`: CoNLL 2006
* `conll2009`: CoNLL 2009
* `conll2012`: CoNLL 2012
* `conllcorenlp`: CoNLL CoreNLP
* `conllu`: CoNLL-U
* `ctsv`: WebAnno TSV v2
* `ctsv3`: WebAnno TSV v3.3
* `dkpro-core-tei`: TEI P5 XML (legacy)
* `dkpro-core-uima-inline-xml`: Inline XML
* `dkpro-core-xml-document`: XML (generic)
* `docx`: DOCX
* `html`: HTML (legacy)
* `htmldoc`: HTML
* `htmldoc-zip`: HTML (ZIP)
* `imscwb`: Corpus Workbench Format (aka VRT)
* `json`: UIMA CAS JSON (legacy)
* `jsoncas`: UIMA CAS JSON 0.4.0
* `jsoncas-struct`: UIMA CAS JSON 0.4.0 (XML/PDF structure)
* `lif`: LAPPS Interchange Format (LIF)
* `mhtml`: MHTML (Web archive)
* `nif`: NLP Interchange Format (NIF)
* `pdf2`: PDF
* `pdf2JsonCas`: PDF (with embedded JSON CAS)
* `pdf2XmiCas`: PDF (with embedded XMI CAS)
* `perseus_2.1`: Perseus treebank XML 2.1
* `pubannotation-annotations`: PubAnnotation Document with Annotations (JSON)
* `pubannotation-sections`: PubAnnotation Document with Sections (JSON)
* `rdfcas`: UIMA CAS RDF
* `tcf`: WebLicht TCF
* `text`: Plain text (**DEFAULT**)
* `textlines`: Text lines
* `tsv`: TSV - Webanno format
* `textlines`: Plain text (one sentence per line)
* `pretokenized-textlines`: Plain text (space-separated tokens, one sentence per line)
* `tsv`: WebAnno TSV v1
* `xmi`: UIMA CAS XMI (XML 1.0)
* `xmi-struct`: UIMA CAS XMI (XML 1.0 + XML/PDF structure)
* `xmi-xml1.1`: UIMA CAS XMI (XML 1.1)


You can find a class with all the formats in `pycaprio.core.mappings.InceptionFormat`:
Expand Down
34 changes: 32 additions & 2 deletions pycaprio/core/mappings.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,11 @@ class InceptionFormat:
DEFAULT = "text"

CONLL2000 = "conll2000"
CONLL2002 = "conll2002"
CONLL2003 = "conll2003"
CONLL2006 = "conll2006"
CONLL2009 = "conll2009"
CONLL2012 = "conll2012"
CONLLCORENLP = "conllcorenlp"
CONLLU = "conllu"

Expand All @@ -20,13 +23,25 @@ class InceptionFormat:

HTML_LEGACY = "html"
HTML = "htmldoc"
HTML_ZIP = "htmldoc-zip"
MHTML = "mhtml"

DOCX = "docx"

LIF = "lif"
NIF = "nif"
PDF = "pdf"

# INCEpTION renamed the PDF format id from "pdf" to "pdf2"; the bare "pdf" id no longer exists.
PDF = "pdf2"
PDF_WITH_JSON_CAS = "pdf2JsonCas"
PDF_WITH_XMI_CAS = "pdf2XmiCas"

PUBANNOTATION_SECTIONS = "pubannotation-sections"
PUBANNOTATION_ANNOTATIONS = "pubannotation-annotations"
TCF = "tcf"

IMSCWB = "imscwb"

PERSEUS_2_1 = "perseus_2.1"
PERSEUS = PERSEUS_2_1

Expand All @@ -37,12 +52,27 @@ class InceptionFormat:
DKPRO_CORE_TEI = "dkpro-core-tei"
TEI = DKPRO_CORE_TEI

XML_DOCUMENT = "dkpro-core-xml-document"
INLINE_XML = "dkpro-core-uima-inline-xml"

UIMA_CAS_BINARY = "bin"
UIMA_CAS_JSON = "json"

# Legacy UIMA CAS JSON format; deprecated in INCEpTION in favour of UIMA_CAS_JSON_0_4_0.
UIMA_CAS_JSON_LEGACY = "json"
UIMA_CAS_JSON = UIMA_CAS_JSON_LEGACY
# UIMA CAS JSON 0.4.0 (see https://github.com/inception-project/pycaprio/issues/17)
UIMA_CAS_JSON_0_4_0 = "jsoncas"
UIMA_CAS_JSON_0_4_0_STRUCTURE = "jsoncas-struct"

UIMA_CAS_XMI_XML_1_0 = "xmi"
UIMA_CAS_XMI_XML_1_0_STRUCTURE = "xmi-struct"
UIMA_CAS_XMI_XML_1_1 = "xmi-xml1.1"
UIMA_CAS_XMI = UIMA_CAS_XMI_XML_1_0

UIMA_CAS_RDF = "rdfcas"

BIOC = "bioc"


class AnnotationState:
DEFAULT = "NEW"
Expand Down