Learning hebrew like a geek

The other day I was thinking about some practical way of learning hebrew, something dynamic that tickles my scientist curiosity. And I thought, wouldn’t it be cool if I could parse one of the hebrew codexes and get the top 20 most frequent words of a chapter or a book? Well this is an attempt to answer these type of questions.

How? #

So I rolled my sleeves and I thought: I will google the Aleppo codex in plain text so I can download it and later use it for analysis. The problem is, I couldn’t find it…

The next challenge then is, how can I produce a plain text version of the Aleppo codex? I thought about parsing a mysword module, which are in reality a sqlite database so that should be easy… but then when I was doing something totally unrelated (probably around the bees or my first batch of mead) it hit me… I have several bibles from the Sword project already installed on my notebook and is all OSS, so… there is probably a python wrapper around diatheke or similar… and sure it is =)

If you want the technicalities, go read my note on How to parse the Aleppo codex and analyze its content in python.

But if you are not a geekie human, you probably are interested only in the results rather than the bits and bytes. This is why I splitted the whole thing in two notes, one more IT related and one more Hebrew related. So here we go, brace yourselves this is probably going to be long.

I am not sure on how to “slice” the codex so the chunks makes sense for an analysis, for instance: per chapter? Per book? Per “stories”? Or even other slicing like Torah, Neviim and Ketuvim. Since it is not clear I will start slicing by books at first.

Assumptions and limitations #

Hebrew has some peculiarities one of them being the vowel pointing. This brings a ton of challenges. For the sake of simplicity I had to stick to a codex that does not include niqqud whatsoever. I am not sure if this is a good approach or not because two different words without niqqud can render the same writing yet have totally different meaning.

Another problem is that there are words that are trivially known like לא (h2834) or אל (h3882); or words that are not translated yet used very much like את (h7073). I have implemented a filter to have the possibility to skip these words. Not because they are not relevant, the Aleph-Tav has a ton of secrets and importance. But there are mixed together some irrelevant tokens considered as words, like opening and closing brackets and parenthesis or similar. So I implemented a list to ignore these words that are rather known or really irrelevant.

Interesting findings #

It was rather interesting to me how fast the repetitions decline on unfiltered words. For instance the most used word is the את (h854) with 7073 occurrences. Yet 30 words later in a row, there is a 90% decline in occurrences, משה (h) with 704 occurrences.

Experiments organized by sections #

Full Tanakh - top 20 words - unfiltered #

Full Tanakh - Top 50 words - unfiltered #

The words from above as the top 20, plus the following words

Torah #

Experiments organized by books #

Genesis #

Conclusion #

The more I try to know, the less I feel I know… This coding-linguistic area is fascinating and intreaguing. I feel like a taxonomy of this area is needed.

This will probaly take a lot of time, which I don’t have. So I want to try to balance between being effective and being efficient. I cannot afford to spend much time with this project but on the other hand the Hebrew language is really really apealing to me. I have yet to find a balance on how to proceed.

As part of documenting things I am going to include some very interesting resources that I have found.

From the time I spent researching I would probably go with the BHSA DB. This is a proposed roadmap:

• Check the ETCBC/course materials for an introduction
• Check the BHSA/bigTables for Pandas examples and learn how to use it
• Reevaluate then

Also, if focus is more in learning Hebrew, check the Parabible website which looks really lean and straigthforward.

Bible in Hebrew Demo #

• Source: https://www.bibleinhebrew.com/bih/BiH_demo.php?uid=10
• Title: Bible in Hebrew Demo
• Captured on: [2022-06-06 Mon]

See, Hear, and Read the Bible in Hebrew - BibleinHebrew.com #

• Source: https://www.bibleinhebrew.com/bih/
• Title: See, Hear, and Read the Bible in Hebrew - BibleinHebrew.com
• Captured on: [2022-06-06 Mon]

Parabible | Genesis 1 #

• Source: https://parabible.com/Genesis/1
• Title: Parabible | Genesis 1
• Captured on: [2022-06-06 Mon]

GitHub - jcuenod/awesome-bible-data: 😎 A curated list of generously licensed Bible data. #

• Source: https://github.com/jcuenod/awesome-bible-data
• Title: GitHub - jcuenod/awesome-bible-data: 😎 A curated list of generously licensed Bible data.
• Captured on: [2022-06-05 Sun]

Text-Fabric versus SHEBANQ - BHSA #

• Source: https://etcbc.github.io/bhsa/mql/
• Title: Text-Fabric versus SHEBANQ - BHSA
• Captured on: [2022-06-06 Mon]

GitHub - ETCBC/course_materials: Contains scripts to learn to work with the ETCBC database #

• Source: https://github.com/ETCBC/course_materials
• Title: GitHub - ETCBC/course_materials: Contains scripts to learn to work with the ETCBC database
• Captured on: [2022-06-06 Mon]

bhsa/bigTablesP.ipynb at master · ETCBC/bhsa · GitHub #

• Source: https://github.com/ETCBC/bhsa/blob/master/programs/bigTablesP.ipynb
• Title: bhsa/bigTablesP.ipynb at master · ETCBC/bhsa · GitHub
• Captured on: [2022-06-06 Mon]

Shebanq / Words Words #

• Source: https://shebanq.ancient-data.org/hebrew/words
• Title: Words
• Captured on: [2022-06-05 Sun]

Shebanq / text [2017] Genesis 30:1 #

• Source: https://shebanq.ancient-data.org/hebrew/text
• Title: [2017] Genesis 30:1
• Captured on: [2022-06-05 Sun]

Jupyter Notebook Viewer #

• Source: https://nbviewer.org/github/etcbc/bhsa/blob/master/tutorial/search.ipynb
• Title: Jupyter Notebook Viewer
• Captured on: [2022-06-06 Mon]

OSHB Read #

• Source: https://hb.openscriptures.org/read/
• Title: OSHB Read
• Captured on: [2022-06-05 Sun]

OSHB Lexicon #

• Source: http://openscriptures.github.io/HebrewLexicon/HomeFiles/Lexicon.html
• Title: OSHB Lexicon
• Captured on: [2022-06-05 Sun]

Bible Online Learner #

• Source: https://booge.eu/
• Title: Bible Online Learner
• Captured on: [2022-06-05 Sun]

J. Ted Blakley — Online Hebrew Resources #

• Source: https://www.blakleycreative.com/jtb/HebrewOnline.htm
• Title: J. Ted Blakley — Online Hebrew Resources
• Captured on: [2022-06-06 Mon]

Tanach.us text files webservice Text files #

• Source: https://www.tanach.us/Pages/TextFiles.html
• Title: Text files
• Captured on: [2022-06-06 Mon]
• Command: wget –user-agent=" Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36" “http://tanach.us/Server.txt?Deuteronomy26:1-1&layout=Text-only&content=Consonants”

Removing Vowels from Hebrew Unicode Text · GitHub #

• Source: https://gist.github.com/yakovsh/345a71d841871cc3d375
• Title: Removing Vowels from Hebrew Unicode Text · GitHub
• Captured on: [2022-06-05 Sun]

shoroshim.pdf #

• Source: https://halakhah.com/rst/shoroshim.pdf
• Title:
• Captured on: [2022-06-05 Sun]

unfoldingWord® Hebrew Grammar — unfoldingWord® Hebrew Grammar 1 documentation #

• Source: https://uhg.readthedocs.io/en/latest/front.html
• Title: unfoldingWord® Hebrew Grammar — unfoldingWord® Hebrew Grammar 1 documentation
• Captured on: [2022-06-05 Sun]