Will the rabbis of the future be robots? The project stirring up a storm in the world of Halacha and technology
The HalachaBench project tested AI models on questions from the Chief Rabbinate's certification exams. Claude Fable 5.1 led with a score of 79.7, but missed about a fifth of the requirements. Rabbi Avraham Stav called the findings 'fascinating'.
A new project called HalachaBench tested advanced artificial intelligence models on real questions from the rabbinical certification exams of the Chief Rabbinate of Israel. The models were required to answer in Hebrew under 'closed book' conditions and without internet access. The leading model, Claude Fable 5.1 by Anthropic, scored 79.7 out of 100. It was followed by GPT-6 Astra with 73.3, GPT-6.1 Sol with 72.8, Claude Opus 5.5 with 69.1, and Gemini 3.8 Flash with 64.4. The free models scored below 11 points. The test was based on 60 questions from five questionnaires in various Halachic fields. Rabbi Avraham Stav called the findings 'fascinating' but questioned the test conditions. The project's creators emphasize that this is an initial intellectual exercise and that no rabbi has yet examined the parameters.