Table of Contents

Technology

XML Databases

It is possible to store the texts as XML files in the file system and use XSLT (as implemented by Saxon) to select requested pieces and transform them into presentation form. Indeed, I’ll have a copy of all the texts in simple XML files anyway, since I need to check the texts into a revision-control system.

It seems likely, though, that I’ll need to store the texts (also) in an XML database. Here are some requirements that make me think so:

  • Access parts of documents in response to a query
  • Fetch fragments of the documents referenced from a given one
  • Find documents referencing a given one (link reversal)
  • Full text search

Only first of these requirements can realistically be satisfied without some indexes. On the other hand, only first two are trivially satisfied by an XML database (like Exist). Integration between Lucene text indexing package and Exist needs to be looked into. As for link reversal, we’ll probably have to write the indexer and accessor ourselves…​

It is clear that a query language to be used is XQuery. It is a nice, functional, non-statically-typed language, that have recently acquired update and text search capabilities. (XXX)

TEIвцы тоже согласны, что надо пользоваться XMLьными базами данных и XQuery [18].

Информацию о различных XMLьных базах данных приводит Bourret. Некоторые бесплатные базы данных для XMLа:

XQuery

Some use XQuery as the (almost) only implementation language for the application (e.g., AtomicWiki). XQuery is a functional language. But XQuery does not have static typesystem or exception processing. I will use Scala as my main implementation language, and XQJ to access XQuery/XSLT processors.

XML and Java

There are APIs for

  • parsing: javax.xml.parsers
  • XSLT: javax.xml.transform
  • XPath: javax.xml.xpath
  • XQuery (XQJ): java.xml.query

javax.xml.xpath only supports XPath version 1

It seems that I can do pipelines using XQJ.

XML Pipelines

Пока что я обнаружил только две системы текстохранилищ ориентированных на TEI: Versioning Machine Versioning Machine и . Обе делали одни и те же люди - Susan Schreibman и Amit Kumar, и обе заглохли. Вторая даже использовала eXist.

Нам надо хранить не только сам документ, но и историю его изменений: кто, когда и что. Это даёт возможность вернуться к любому состоянию, посмотреть историю, заблокировать слишком быстрое изменение текста и т.д. Дя этого надо прирастить к базе данных готовую version control system (XXX не очевидно), а именно - GIT.

К текстохранилищу должен быть доступ через сетевые протоколы, а не только через web-интерфейс.

Look Into

AtomPub, WebDav, REST, XML-RPC, XML:DB, GIT, Atom, RSS, Citizendium, Annotea, Collate/Anastasia, XForms

URLs

XPointer in the URI, not in the fragment! No delimiters, just URI parts - which can be implicit (not "chapter=3", but "chapters/3", or just "3")! Editions in the URI ("Chumash/boston+toronto/Genesis")! Metadata ("about"), raw XML etc. - in the URI, not as query parameter ("Genesis/about", "chaters/1/raw")! More URI promotion: natural references ("Genesis/2:1", "Genesis 2:1")! Intervals ("Genesis/2:1-3")! Concatenation ("Genesis/2:1-3;5") probably shouldn’t be done through URIs!

Books URIs:

/books/Tanach/editions/…​/[parts/…​/]books/…​/[weeks/…​]/chapters/…​/verses/…​

editions: a | a+b (side-by-side) | a-b (differences)

parts: Torah | Neviim | Ksuvim

books: Genesis | Ionah | …​ (appropriate for part if present)

weeks: Genesis | Noah | …​

chapters: n | m-n

verses: n | m-n (can be present only if one chapter is selected)

Alternative names may be used.

URL may be truncated.

Parts of the URL may be implied - and need to be derived.

Metadata

Metadata is used to:

  • guide navigation
  • provide listings and names
  • create classifications (links)
  • stitch together data directories
  • store application-specific metadata

Some of the data in it has to be duplicated in the text document (for self-containment, and for non-position-based navigation).

We need to be able to handle things like "Chumash/books/Genesis/weeks" and "Chumash/weeks" with one metadata document…​

Locators for the navigational steps can be: - subdirectory/file - element XPATH - milestone XPATH

1) I need to be able to provide a list of selectors (book name/ chapter number/ verse number etc.) on any level.

2) A selector can have multiple names, which I do not want to duplicate (and maintain) in each edition of the text. So, selector names have to be part of the metadata.

3) A text can have multiple structures.They are important for the metadata also. Restructuring of the text is done by XSLT. It seems logical to use the same for the restructuring of the metadata.

It follows that the metadata needs to be processable as XML (and have format similar to the texts). Do I also need it to be processable (in part) as Java objects (using JAXB) - is not clear.

We are going to use milestones [?] to represent multiple structures.

<book n="Genesis">
  <chapter n="1">
      <week n="Genesis" milestone="begin"/>
      <paragraph type="open" milestone="begin"/>
      <verse n="1">
      ....
      </verse>
  </chapter>
  ...
  <chapter n="6">
      <verse n="1">
      ...
      <week n="Noach" milestone="begin"/>
      <verse n="..">
      ...
  </chapter>
</book>

Tanach Markup

What are the TEI-appropriate tags for Tanach?How do we represent the paragraph in the middle of the verse?

Super-Wiki

Wiki with multiple formats ⇒; function reversal (TEI→;HTML; edit; back)…​

Wiki page rename and links correction - if the wiki itself is in an XML database (AtomicWiki) with our link-reversal index, wouldn’t it be easier? History will be kept by the revision-control system…​

Navigation:

  • expand/contract viewport
  • move viewport
  • switch to a different structure preserving focus (from "lesson" to "chapter" in Tanya, for instance)
  • switch to a different edition / look around at editions

Notes

crowd-sourcing TEI files

Web-based IDE with WebDAV’s versioning

BUGS

Upstream:

Sebastian:

  • File a bug against FO stylesheets (title, table of contents).
  • File a bug about reference shape consistency.
  • File a bug about use of @name for reference.

Saxon, Tomcat and relative URIs for the stylesheets. XQuery Server Pages (and eXist).

space before a word that has read/write annotations (Psalm 60)

Styles of biblio references.

Google SSO. GData. RSS/Atom - second edition? Hacking…​?

Start working on XSLT: Genesis → FO

leningrad-import:

  • remove stylesheet link
  • add TEI P5 All declaration; namespace(s)
  • makaf

XProc

Discussions as text.

Convince CiteULike to make their XHTML really XHTML, or at least - well-formed XML. Better - parse RIS.