Smart IDE Editor for Oracle Bone Character Input

Basic Settings

Line Arrow Synchronize 1 Streamline Icon: https://streamlinehq.com

Font:

Font Size:

Subtract 1 Streamline Icon: https://streamlinehq.com

19

Add 1 Streamline Icon: https://streamlinehq.com

Character Spacing:

Subtract 1 Streamline Icon: https://streamlinehq.com

-0.2

Add 1 Streamline Icon: https://streamlinehq.com

Line Height:

Subtract 1 Streamline Icon: https://streamlinehq.com

26

Add 1 Streamline Icon: https://streamlinehq.com

Scroll Speed:

Subtract 1 Streamline Icon: https://streamlinehq.com

1.5

Add 1 Streamline Icon: https://streamlinehq.com

Documentation

🚀 Editor Introduction

Jingyuan Oracle Bone Smart IDE Input Editor on this page, is a powerful text editor designed for the Shang oracle bone script system. It has implemented several features, including strict-loose standardization pair auto-Matching, word and idiom auto-completion, pinyin input, component input, quick link navigation upon hovering, on the basis of the Oracular font database. The implementation of the IDE editor is done on top of Microsoft's Monaco Editor. The latter is the underlying codebase of the mainstream code editor vs-code, the initial design of which is to provide an easy, open-sourced technical support for the editing of various programming languages. The major features of monaco includes language syntax highlighting, type hinting, code auto-completion, line folding, which are widely applied and appreciated in the programming world. The current editor interface underwent further development on top of Monaco library, migrating part of the basic functionalities of the original library, to adapt to the long existing situations in the oracle bone research where the typing of characters and design of graph database had been extremely challenging, and the studies of character graphic forms, phonetics and semantics were decoupled. The IDE has achieved support for the quick input of oracle bone characters, through which the users can type characters into the computer documents , while using codepoints from the Oracular Font Database, thus increasing the research productivity, reinforcing the visual and semantic memorization of relevant characters and familarizing oneself with the complex graph relations and decipherment results in the oracle bone studies.

The major features implemented so far include:

  • Word and Idiom Auto-completion via Speical Trigger Characters: The current IDE has incorporated the funcationality of "word auto-completion", which can be triggered under several special scenerios, including the triggerings from three different typing modes:
    1. Default Mode: This mode is active when typing normally in the editor. There are two main types of auto-completion scenerios: One is the strict-loose standardization auto-completions, which will automatically complete the strict and loose standardization forms of the same character based on the pre-defined strict and loose pairs in the backend database. This mainly mainfests as {strict} loose form, such as when typing "鼎" will automatically prompt "鼎{貞}" completions, and typing "惠" will prompt "叀{惠}" completions, which then user can select and substitute the text (or don't, depening on the need). The auto-completion pairs implemented so far have exceeded 1,000, which can partially reflect some of the standardizing norms generally accepted by the academia. Needless to say, the implementation of this IDE editor can help significantly save time in searching for previous text examples and finding characters. Another auto-completion use under the default mode is the common word completion, which automatically matches the pre-defined common words, such as the stems and branches, multiple-character words, etc. For example, typing "癸" in the editor will automatically prompt "癸丑", "癸亥", etc. and other stem and branch completions. This mode is mainly to provide quick access to some common words, reducing the need for character finding and typing process.
    2. @substitute_mode: This mode is triggered by @ (an arbitarily defined character), which is mainly used to provide some alternate forms of the same character. The alternate forms include different "standardizations", as well as the original forms of the parent-level "Graph", "Subgraph" for the character. For instance, one can type @焚 in order to get its subgraphs “𤆰”, “򴯜” in modernized forms, or its archaic forms like “󰝋”、“󰝥”. This mode can beeter adapt to the academic divergences regarding the grouping of graphs caused by different decipherment research. When in doubt, one can select "subgraph" to display a character form, which can more precisely reflect the appearance as seen in rubbing copies, and it can also provide support for the indexing and querying of "subgraphs" in future database development. And another usage under this special mode is that @ can trigger pinyin input, which can locate and prompt the relevant characters based on its Chinese pronunciation. For instance, typing @zhu can prompt the character "朱", as well as the word auto-completions like "不 򧄾 鼄" and "󰽢" (竹). For more examples, try typing @shu can return characters "򡮉", "򡭪", "𠅩", which is because the latter ones are the standardized forms of the sub-level characters under "孰". In the current IDE environment, as long as a specific standardized form locates inside the CJK plane (including extension planes), typing the pinyin pronunciation for it will return and prompt the correct character. Evaulating by this functionality alone, the current IDE can be said to have achieved a traditional pinyin input tool.
    3. %component_input: This mode is triggered by special character %, in which % signifies character division, i.e. one character can be divided into components which can be used to input that character. The primary result of this mode is allowing the filtering of relevant characters based on their component composition, which funcations by checking the typed components against the pre-defined component encodings for each character. For example, typing %日月 in the editor will return characters like "明", "朝", etc. which contain the components "日" and "月", or typing %人人人 will return the character "眾". This mode is essentially a traditional component input tool, and it is more powerful than the above two modes in general. It can extend to and cover the typing complex characters that do not have fixed standardizations, or whose pronunciations are unknown, but have common components. It is generally applicable to various single-component, quasi-single, and other compound characters, and can be used in conjunction with the above two modes to adapt to the overall input problems of oracle bone characters discovered so far. Moreover, gradually familiarizing with this feature can help users better understand the character composition rules of oracle bone characters and improve one's learning efficiency.

Here lists some more practical auto-completion examples:

  • @jian --->
  • @mian --->
  • @qu --->
  • %虍人 ---> 󂉷
  • %木木刀 ---> 󱝚
  • %鹿文 ---> 󲞍
  • 翌 ---> 翼{翌}
  • 惠 ---> 叀{惠}
  • 牢 ---> 𫳅{牢}
  • 遘 ---> 冓{遘}雨
  • 用 ---> 𢆶{茲}用
  • 余 ---> 余一人

To some extent, the three typing modes mentioned above can correspond partially to the standardization query, pinyin query, component query implemented in the graph database page. The IDE editor is basically integrated with a real-time database query functionality with no query delay. Users can quickly switch between the three modes according to practical needs, and use in conjunction with the graph database interface to achieve the maximum effect. Naturally, the specific features of this editor have to be adjusted based on practical needs. Therefore, suggestions and feedback are quite welcome to further improve the current interface.

Warning Circle Streamline Icon: https://streamlinehq.com
One thing to note is that the current component encoding work has only be done for 2/3 of all characters. The remainder of the work is in active progress, which is estimated to be complete in the near furture
  • Data Hint upon Hovering: In the case where there are existing characters in the IDE editor, users can try hovering the mouse over any character, and the editor will automatically trigger an instant database query based on the content of the character located at the cursor position, and display the character information returned from the backend. Moreover, the character information also provides a quick link navigation feature, which allows users to click on the blue link in the prompt to jump to the dedicated character page in the graph database, and check more character and group information. This feature, when combined with other pages on the website, can truly achieve seamless connection between text input, text retrieval, and character learning. However, it should be noted that the current feature has a certain delay (~3s) built in to ensure that it does not trigger frequently when typing or hovering the mouse quickly, but this may also cause a noticable delay in the normal display of data, so users should be aware of this when using it. In addition, the triggering of the data display is determined by the specific content of the character at the cursor position. If the character being hovered over is not included in the database, no information will be returned.
  • Semantic Highlighting: The editor has implemented some simple text highlighting rules for oracle bone inscriptions based on some grammatical regularities. Currently, the main highlighting rules include [...] missing character completion, {...}loose standardization form, and a few other simple rules. These texts are not part of the original text in principle, and are difficult to distinguish from the original text in the static state. Therefore, visually displaying them in color would make it much easier to distinguish them from the main oracle bone text. In addition, the editor also has a symbol highlighting feature, which can be triggered by the Ctrl+F text search function, which will highlight the matched text. In addition, you can also select a specific text or word, and then use the Ctrl+F2, Ctrl+F3 shortcuts to quickly highlight all similar symbols.
  • SideBar Pasting: The editor's left sidebar pane is set up with a component table that provides a full list of oracle bone components and some common characters' character code tables. Users can try clicking on the relevant buttons, and the editor will respond by pasting the selected text at the original cursor position in the editor, and trigger the corresponding completion action. This feature can provide more efficient and wider character codepoint support when some oracle bone components do not have related standardizations, or are difficult to input. The design principle of the component table is the same as the component query in the graph database page, both of which use the latest natural classification method for oracle bone components by Prof. Huang Tianshu. The components are arranged in the precise order of "nature", "human", and "object", and the specific components are slightly modified based on the actual character compositions when encoding the components. Efficient usage of the functionality can be achieved after slight familiarization with the module.
  • Rich Shortcut Key Support: The editor is based on the native monaco library, which provides a rich set of built-in key combinations, which are generally similar to the vscode keybindings. In addition to common word processing shortcuts such as Ctrl+F search, Ctrl+Z undo, Ctrl+Y redo, etc., the editor also provides support for some special shortcuts, such as Alt+↑ or Alt+↓ for quick text line breaks, Ctrl+Shift+K to delete the current text line, Ctrl+Shift+↑ to copy the current text line, etc. More shortcuts can be viewed by triggering the command panel with Ctrl+Shift+P. Familiarity with these shortcuts can greatly improve input efficiency and free users from frequent mouse operations. In the future, the editor can also provide more customized shortcut settings if needed.
  • Auto-Matching and Closing of Brackets: As a popular feature in common code editors, the editor has also migrated and integrated the functionality of automatically matching and closing brackets and quotes. For example, when typing various parentheses, brackets, quotes and other symbols, only one symbol (left bracket) needs to be typed, and the editor will automatically match and complete the other bracket. This small feature, although seemingly simple, can greatly improve input efficiency when maintaining the input state for a long time. The current supported bracket matching symbols include {}, [], 【】, (), (), <>, "", '', “”, ‘’, etc.

Apart from the above-mentioned powerful features, the contemporary mainstream code editors such as vscode, vim, atom, etc., have already been developed and improved for many years, and have many more hidden utilities and features (such as code formatting, code folding, code snippets, AI programming assistants, etc.). Adding to these include also the strong support and community power in the programming world. Therefore, a myriad of plugins and extensions are constantly emerging each year, providing better and better editing experience for coding languages. And as an underlying open-source codebase, the Monaco editor, although relatively less advanced in features and plugins than vscode, etc., due to its low-level design principles, the usage of its API interfaces can prove quite universal, and it does not have any cognitive presets for the rules of the language itself (so there is no fundamental difference between programming languages and natural human languages in essence), developers can just adapt to the application needs of their own languages according to the instructions in its official documentation. In this sense, transplanting the principles of code editors to the Chinese language, especially to ancient text editing, is actually a very reasonable method, and can be categorized as an interdisciplinary effort. Using the three-in-one architecture and coupling of oracle bone font/database/IDE, it can be said that this has the potential to completely solve the problem of "pasting thumbnails" that the oracle bone community has faced for many years, and can also correct the current over-focus on the development of traditional input tools. Of course, the release of the current IDE editor is limited by the release schedule and various related work of my personal doctoral thesis, and therefore I have only explored the tip of the iceberg in an IDE's potential. The main purpose of the current tool development is just to support the input of character code, so there are not too many complex functions unrelated to character typing. And, the cost and efforts needed for collecting data are also very expensive, so the current purpose is only to support inputting and typing. But in the future, benefiting from the power of the community, text mining, text clustering, visualization, and even AI-supported built-in text analysis models can be introduced gradually. Based on the versatile functionalities provided by front-end programming, the implementation of these functions is not quite difficult, but it will indeed require siginificant time and effort investment.

✨ Usage Tips

The design of this tool is based on the principle of simplicity and ease of use, but there are still some things to note in actual typing, mainly in the following cases:

  • Note the difference between strict and loose standardization: The current oracle bone studies tend to use the character form and appearance to determine the specific standardization, so many characters would have both strict and loose forms, and sometimes these are mixed and interchangable, making it difficult to typing oracle bone inscriptions correctly in the document. In view of this, the @substitute mode built into the tool has a quick conversion between simplified and traditional characters, such as entering @马 to get the character "馬", and entering @贞 to get the character "貞". This effectively avoids the switching between simplified and traditional characters via one's installed input tool, and can maintain a relatively continuous use of the same input mode. However, it should be noted that the default input mode strictly adheres to the principle of strict-loose standardization, so if you encounter some characters that cannot trigger the match, you can first check whether the character meets the default triggering principle. For example, if you enter the simplified character "灾", you cannot trigger the completion of "巛{災}", but only the completion of 灾{烖}, this is because the two oracle bone characters's modernized forms are not actually the same, so the IDE has also been configured differently in the data annotation process, which is one thing that needs to be noted in particular.
  • Note the difference between components for a graph and a subgraph: The oracle bone characters are divided into two primary levels of "graph" and "subgraph" in the current database, and the "graph" is the main character, while the "subgraph" is the character or characters that are sub-level units of the main character. The current %component_input mode is mainly responsible for returning "graphs" or "subgraphs" with specified components, and does not include, in principle, the alternate forms of the characters. This can sometimes lead to the situation where using a specific combination returns only the "subgraph" character, and not the "graph" character, or sometimes there are known unique forms of a character, but as it is classified as a "subgraph", it can not be located when typing the relevant components. For instance, the character 彘 has a known variant form "󲀋" ("subgraph" level), and typing %矢豕 will not directly return this form, but only the character 彘 in "graph" level. This is especially so if the "subgraphs" have the exact same components as main "graph", with perhaps only different spatial relations, then there will be not need at all to encode the "subgraphs" separately, in consideration of saving the database size and editor JavaScript memory heap size. Naturally, ideally the returned results from component input could include all "subgraphs" as well based on their relations to main "graph" level, which is not particularly difficult to achieve either. However, we should consider the overall situtaion where the number of oracle bone graphs and variants are quite high, and the occurrences of some of the components in these characters are extremely frequent. For example, under current implementation where only main "graphs" are encoded with components, typing %日 still will return hundreds of possible results. Including "subgraphs" will only return thousands more results in one prompt window, thus defying the initial design purpose where %component_input is used to filter the graphs in the first place, not to return a bloated list of unrelevant characters.
  • Use common character forms to trigger: In the study of oracle bone inscriptions, there are various complex characters with intricate structures and strict standardization forms. If one is too focused on directly finding and typing these characters, the results may not be ideal. At the same time, these complex characters often have a more common standardization form in modern Chinese, such as "𢦏{災}", "򮙈{害}", "灾{烖}", "𦎫{敦}", etc. (the latter of which are loose standardized forms), all of which contain at least one common standardization. Regardless of which standardization is entered, the auto-completion can be triggered in the @ mode. Therefore, when encountering complex characters one is uncertain how to input, consider first whether there are alternative ways to input the character that will prompt for auto-completion, which can speed up the input of complex texts.
  • Use multiple modes together: The three input modes of the current tool are designed to be distinct from each other, each with clear responsibilities and covering a range of scenerios. The default mode is only responsible for providing strict-loose standardization matching and some common word and idiom completions. It overall strictly adheres to the general rules of oracle bone inscriptions and transcribing norms, making it easier to produce accurate oracle bone inscriptions. The @substitute mode is responsible for providing quick pinyin input and alternate input for different forms of the same character (including "subgraph" and original archaic forms), and does not follow very strict rules, but is designed to filter the characters more loosely, providing fast visual aid. The %component_input mode is, on the other hand, designed to provide powerful filtering and retrieval based on component encodings, and is capable of covering and indexing all oracle bone characters for searching, and is generally more accurate than the other two modes. Therefore, understanding the differences between the three modes, when encountering complex characters, one can use a mixture of pinyin input, component input to find the character, and then use the loose standardization to trigger the completion that would be more professional according to the norms set and used in the academia.

📝 Text Copying Issues

Although in principle, the oracle bone text editor on this page uses unicode points as sole basis, which can therefore adapt to the rendering and display of various complex oracle bone modernized and archaic forms. Users can switch dynamically between these forms in the "font" option directly under the IDE editor. However, it should be noted that this functionality is only available in this website. Once outside the website system, such as when trying to copy the text into a local word or excel document, the correct display of oracle bone texts is still a tremendous problem. In order to display these complex characters (in terms of unicode points, these characters cover the standard CJK, CJK Extension A-I, BMP, PUA-A, PUA-B, etc. Unicode planes), the website actually integrates multiple fonts such as "Oracular font", "seal font", "HanaMinA", "HanaMinB", "KaiXinSongB", etc., each with their own responsibilities for displaying a range of the characters, in order to maintain the website's support for oracle bone characters and their standardizations.

However, one thing to be emphasized is that all these fonts are either commercial fonts or private research achievements, and are not included in the default fonts of general computer systems. Therefore, when copying the texts to a local file, it is inevitable that there will be problems such as garbled characters, missing characters, or wrong font substitutions. To solve these problems reasonably, some additional settings are needed. Taking word software as an example, if one wants to display oracle bone inscriptions in the document, you need to go through the following steps:

  1. Download and install the corresponding fonts. This has been explained on the font download page, users can download the required fonts according to their needs, and then install them into the local system. If you want to display all the characters, except for the oracle bone font, the other mentioned seal fonts need to be downloaded and installed. After installation, you need to close word and reopen it to make the new font take effect (no need to restart the computer in windows).
  2. Copy the text in the editor to word. This editor is a plain text editor, without the application any text styles and images, so when copying and pasting into word or excel, only the text content will be copied, without the original font settings, so you would generally need to go to the next step.
  3. Select the text and set the font. After pasting the text into word, depending on the font you want to display, if archaic form first, i.e. you want to display the oracle bone characters in their original archaic forms, the processing is relatively straightforward and you can use the "Oracular" font provided by this website, and replace the text to be displayed with the "Oracular" font to display all the characters correctly. But if it is modernized form first, i.e. you mainly want to use the modern standardized Hanzi form to display the text, then the processing is more complicated. You would need to begin from most basic fonts: first select an English font, such as "Times New Roman", to ensure the styling of English characters and symbols is correct (can be omitted if there is no need to configure for English texts), then select a basic Chinese font, such as "SimSun" to display the common Chinese characters, and then select the Hanzi character set fonts, such as "seal" (seal font is preferred in terms of order of loading fonts, because for some reason, seal will be recognized as an English font), "HanaMinA", "HanaMinB", "KaiXinSongB", etc., so that only some garbled characters will be left, usually displayed as missing characters such as , , , etc., these are the unique character codepoints of the "Oracular" font, and you need to select "Oracular" for each character one by one to reproduce the original effect of the editor. The advantage of this method is that you can choose different fonts according to your actual needs, but the disadvantage is that the operation is much more cumbersome, and it will be time-consuming when replacing a large amount of text.

Generally speaking, the display of oracle bone inscriptions in the absence of standard unicode support is still quite complex. General office software such as Microsoft Word, Excel, PDF, or WPS Word can directly use the font selection in the top menu bar to change the font type displayed in the document. The advantage of this method is that it is simple and easy to understand, but the unreasonable font replacement mechanism will also lead to a lot of operations when multiple fonts need to be used to display the same codepoint, and batch modification of fonts later will cancel the original selected specific font, which is not convenient for publishers and journal editors to perform unified typesetting, which is a defect in the design of office tools themselves. In comparison, other professional text layout tools such as Markdown, LaTeX, etc., use tags based on html or xml format to define font types and styles, which is more reasonable. These tools usually only need to define the required font via a global stylesheet at the beginning of the document, and can be modified at any time, and their typesetting capabilities are greater than that of applications such as Word in some aspects. However, these tools have a steeper learning curve, and are not widely applied at present. But for those who are familiar with this file type, the following methods can be considered.

In the case of a Markdown type file (it is recommended to use an editor that supports html styles such as Obsidian), the following code can be used to define the font:

Inscriptions.md
<span style="font-family: 'Times New Roman', '宋体', 'seal', 'HanaMinA', 'HanaMinB', 'KaiXinSongB', 'Oracular';">
Enter oracle bone text here
</span>

而如果是原形优先,则需要调整字体优先顺序:

Inscriptions.md
<span style="font-family: 'Oracular', 'Times New Roman', '宋体', 'seal', 'HanaMinA', 'HanaMinB', 'KaiXinSongB';">
Enter oracle bone text here
</span>

Using the above settings, you can achieve complex display results. This procedure actually relies on the font fallback mechanism, which means that when the current font cannot display certain characters, the next font will be automatically called to display them, which can achieve a chain of font substitutes automatically. The design of this website also fully utilizes this mechanism, i.e. in the first case of modernized form first, "Times New Roman" is used to display all English characters, and any Chinese characters that cannot be displayed by Times New Roman will be replaced by "SimSun", "SimSun" can only display basic CJK characters, so the extension and BMP characters will be tried to be displayed by "seal", "Hana", etc., and the PUA characters that "seal", "Hana" do not use will be displayed by the Oracular font. In the second case of archaic form first, the situation is similar. But one may notice the Oracular font can also used to display basic CJK characters. This is due to the multi-codepoint design of the font, i.e. a character will have two corresponding codepoints from CJK and PUA areas at the same time in the font, if the font defined in the first place (higher priority) it will then be used to display the CJK character. This is an important step in the font design, which to some extent avoids the need for external codepoint mapping tables for displaying common CJK characters but in archaic counterparts.

In general, the input of oracle bone inscriptions has been a very difficult problem for many years, and even with the current technical means, coupled with various programming methods, it is not possible to completely solve it. One of the main reasons is due to the lack of a unified Unicode solution for character encoding, and oracle bone inscriptions have long been considered challenging to be incorporated in Unicode. The main reason for this is that the academic community has no unified agreement over some of the divisions and mergings of characters, encoding rules, character form standards, and input methods, and that the production of related fonts also has insurmountable costs and knowledge barriers. Essentiallt, fonts and character encoding are two sides of the same coin. Character encoding without proper font support is just a pre-assigned number in the computer, without a font engine to render it - a blank space. And without a unicode standard, the specific character allocation of a font will ultimately be quite unstable, and will vary over time or due to different font providers, and cannot be shared by the public. Therefore, the difficulty in displaying oracle bone characters is actually due to multiple uncontrollable factors, and is not something that can be solved by a simple font library or input method. The same also applies to other systems such as bronze inscriptions, seal scripts, and Warring States scripts, and ultimately relies on the accumulation of academic research results and related technical support of Unicode to have hope of solving the problem.

⚠️ Known Issues

During the debugging process of the current tool, some practical situations were encountered that were not handled properly by the native monaco API, and part of them were corrected, and the current tool can adapt well to normal typing mode, pasting mode, text copying from sidebar component on this page, and mouse-movement triggering, but it should be noted that there are still several input situations that may not reached ideal conditions:

  • Incorrect special symbol order will not trigger prompting. Under the special modes such as prefixed content by "@" and "%", the correct triggering text mode is @text or %component, which means that the "@" and "%" must appear before the text to be replaced, and it is best to define them on the same line. If you first enter text and then move the mouse to add "@" or "%" in front of the text, the prompt will not appear. This is because the current program's coding logic is to extract the text in front of the cursor upon typing as the actual detected text, and any text behind it will be automatically ignored.
  • Cannot trigger when multiple special symbols exist. In the "@" and "%" modes, there can only be one "@" or "%" on the same line in the editor, and if there are multiple "@" or "%", the prompt will not appear, so when editing, try to delete redundant trigger symbols.
  • Different input modes cannot trigger continuous triggering. The text prompts received from the special modes "@" and "%" will normally replace the original text, but as new text, they generally will not trigger another round of auto-completion under the default mode. This is the default behavior of the monaco native API, and the current IDE has not been modified it during development. The purpose of this is to avoid overly complex programming logic and recursive loops (otherwise, if the component triggers the single character, the single character triggers the strict-looose pairs, the strict-looose pairs trigger the word entries completions, the word entries trigger the sentence examples, the sentences trigger the paragraphs, it will cause the triggering boundaries to be unclear, causing possible memory overflow and abuse). Therefore, when using, if there is need, you can manually trigger recursive completions by copying and pasting the new text.
  • A few non-CJK character components cannot trigger the component formula. This situation includes characters like "򶫴", "򧕤", "򧈟", etc., although these characters like "򶫴" have recognizable components like "𡆥", "犬", and the database has indeed been encoded accordingly, but using %𡆥犬 still will not return character "򶫴". This is because characters such as "𡆥", "򲤶" which are outside CJK plane are rendered as one character visually, but they occupy in effect two character spaces. These characters are known as "surrogate pairs" in the unicode world, which have very unique processing rules in text processing and are often treated as if there are two separate characters. It is also due to this consideration, at least two characters before the cursor position will be extracted and tested against parsing rules upon each default typing operation, and was done so recursively among the two character combinations (it is also because of this, matchings like 㘡{上甲} can be processed). On the other hand, under the "%" component input triggering mode, the current IDE will try to perform matches using a single character as a unit. Therefore, in actual typing, it is recommended to try to divide components as small as possible for characters such as "򶫴". For instance, "𡆥" can be further deconstructed into "囗" and "卜", therefore %卜犬 can also trigger "򶫴".
  • Incorrect display of characters in the completion list in prompt window. This situation is also common in the case of "surrogate pairs", such as the character "𫡜", which will display "��{次}" in the completion list of the prompt window, or in the case of word auto-completions, "򢀭{殞}" will be displayed as ��{殞}, which seems to be a character error, but it is actually the expected result. This is because the monaco underlying API processes the character triggering range in the same way as the input character range (both are two characters), so when rendering, characters such as "𫡜" are split in half. The split part of the codepoint does not conform to the utf-8 standard, so it is displayed as garbled text. This type of text in the utf-8 encoded .json file also have similar issues. But these are only temporary rendering effects in the drop-down menu only, and if the prompt is accepted, the completion in the editor should be displaying as normal, so it does not affect normal typing.

The above few issues are some of the observations made during testing and debugging. In addition, due to the complexity of data structures, programming logic, and the various program bugs that may exist in the native Monaco API itself, the tool may have some additional issues that were not discovered during development, so if you encounter problems, official feedback would be welcome.

💻 Programming Guide

In terms of employing the method of using professional programming IDE editor to develop an intelligent oracle bone text input method, this tool currently may be the first of its kind. Although the implementation seems relatively complex, it actually is quite easy by fully utilizing the powerful code base and native APIs of the monaco-editor library. However, one note that should be raised is that the oracle bone script as a unique natural language, the complexity of its character relationships, the instability of graph categorizations and component encodings, and the diversity of related data structures will lead to many unknown logical pitfalls, when compared to general programming languages such as python, C, javascript, vue, etc., affecting the accuracy of the IDE functionalities. Therefore, how to make proper adjustments to the specific programming logic that reflect the lingustic characteristics, adapt to the existing literature in the field, and take into account the feasibility of practical applications, is no doubt a quite essential research topic. Due to this consideration, this section will be dedicated to demonstrating the steps for implementing an IDE based on Monaco library, and how to use the its native API functions to adapt to the uniqueness of the language as a proper research tool. Some of the specific solutions for the current text editor implementation may be adapted to scripts such as bamboo script, bronze script, etc. as well. And it is hoped that this guide will inspire programming enthusiasts and digital humanities researchers in building more productivity-enhancing tools. (Note this section will contain some JavaScript code, and requires a certain understanding of the web front-end development process, if the reader is not interested in this, the section can be skipped).

First, for the installation and usage of monaco-editor in a web front-end environment, one can refer to the official docs of monaco or github for detailed instructions. The editor's use requires a JavaScript and node.js framework (such as vue, React, or Angular.js) to work properly. And in the absence of a front-end server that could supports JavaScript, you can also use the demo page provided by monaco-playground for relatively simple language testing.

Registering a custom language: The default languages supported by the Monaco editor include python, javascript, typescript, json, html, css, and dozens of other programming languages, which is indeed a lot coverage. However, as our languages such as oracle bone script is a natural human language, it is naturally not included in the existing programming language definitions. Therefore, in Monaco, we need to register a custom language using the monaco.languages.register function, to let Monaco know that we are starting with a custom language with a blank slate, as shown below:

Language Definition.js
monaco.languages.register({ id: 'jiaguwen' })

This is to instruct Monaco that we are to use a brand-new custom language. However, this alone does not inform Monaco of anything such as what are the grammatical rules related to this language, so we also need to register some syntax and lexicon-related content.

Defining language syntax: After defining the language id, we can choose to define some syntax rules for the language, so that Monaco knows how to highlight specific character patterns and combinations, such as [...], {...}, <...>, etc. Using these bracket pairs as an example, we could try to highlight these patterns using the setMonarchTokensProvider(lang_id, options) function:

Language Definition.js
monaco.languages.setMonarchTokensProvider('jiaguwen', {
  ignoreCase: true,
  unicode: true,
  tokenizer: {
    root: [
      [/\{.*?\}/, 'tag'], // highlight content inside brackets such as {貞}
      [/\[.*?\]/, 'variable'], // highlight missing character completions inside [...]
      [/[□|�]/, 'attribute.value'], // highlight special missing symbols (optional)
      [/\s+/, 'whitespace'], // highlight empty symbols like spaces. Default is not to highlight at all. But this behavior can be overwritten in setting `renderWhitespace: ""all"` to force spaces to be highlighted.
    ],
  },
})

After some initial steps of configuration above, we can have the following highlighting effect (of course, we can also customize what color to use for each token):

monaco-editor 语法高亮

The tokenizer rule shown above in the code segment is based on Regular Expressions for define specific patterns. The so-called grammatical "tokens" can be understood as the "grammatical particles", "grammatical elements", or "grammatical rules" in the natural languages. The relevant naming of the tokens can be inherited from Monaco's default theming guide (of course, one can just name them completely randomly), some of which can be found in the vscode guide. However, it should be noted that Monaco does not support all tokens used by vscode, and sometimes defining them according to vscode defaults will not have any effect whatsoever on the syntax highlighting. Moreover, it is also quite troublesome that the official Monaco docs do not provide a complete token list either. However, after some repeated testing by the author, it can be recognized that the following tokens can have a certain impact on color highlighting:

token list.js
const monacoTokenGroupA = [
  'tag',
  'metatag',
  'keyword',
  'tag.html', // html tags
  'delimiter.html', // tag brackets
  'metatag.content.html',
  'tag.css',
  'keyword.css',
  'keyword.ts',
  'keyword.js',
  'keyword.flow',
  'meta.scss',
  'string.sql',
  'strong',
  'emphasis',
  'keyword.field.marc',
]

const monacoTokenGroupB = [
  'attribute.name',
  'attribute.name.html',
  'attribute.name.css',
  'string.key.json',
  'predefined.sql',
  'delimiter.bracket.css',
  'delimiter.bracket.ts',
  'delimiter.bracket.js',
  'class',
]

const monacoTokenGroupC = [
  'attribute.value',
  'attribute.value.html',
  'attribute.value.xml',
  'attribute.value.css',
  'keyword.flow.scss',
  'string',
  'string.html',
  'string.css',
  'string.js',
  'string.ts',
  'string.value.json',
  'delimiter.xml',
  'attribute.value.subfield.marc',
]

const monacoTokenGroupD = [
  'number',
  'attribute.value.number',
  'attribute.value.unit',
  'number.hex',
  'number.float',
  'number.js',
  'number.float.js',
  'number.float.ts',
  'number.hex.js',
  'number.octal.js',
  'number.binary.js',
  'number.ts',
  'attribute.value.hex.css',
  'attribute.value.number.css',
  'attribute.value.unit.css',
]

const monacoTokenGroupE = [
  'type',
  'variable',
  'title',
  'parameter',
  'property',
  'identifier.ts',
  'identifier.js',
  'tag.id.pug',
  'tag.class.pug',
  'type.identifier.ts',
  'string.yaml',
  'keyword.json',
]
const monacoTokenGroupF = [
  'constant',
  'regexp',
  'member',
  'punctuation',
  'metatag.html', // IDoctype
  'metatag.xml',
  'metatag.php',
  'delimiter.parenthesis.ts',
  'delimiter.parenthesis.js',
  'delimiter',
  'delimiter.js',
  'delimiter.ts',
  'operator',
  'operator.scss',
  'operator.sql',
  'operator.swift',
  'punctuation.backslash.marc',
]

The groups like monacoTokenGroupA are defined based on author's arbitrary rules. It is recommended to use same or similar colors for the same groups. Doing so, it can avoid color overloading for visual display issues. Of course, not all tokens above are suitable be used in the oracle bone script, and it is recommended to use as few token groups as possible to avoid too many different colors. Many tokens there are highly language-specific, and will only take effect in some specific languages like js, sql, or html, which naturally are no concern of ours. For natural languages like oracle bone script, it is enough to set just about ten token types to cover specially patterned text content (such as missing characters, direction markers, completion brackets, and subordinated brackets), and the token names can be randomly selected from the above list. Too many color settings will make the editor's visual effects too extensive, which is not conducive to long-term editing.

Auto-completion item registration: The Auto-completion function is probably the main feature we want to pay special attention to (Official demo provided by Monaco provides a simple auto-completion example for the json language). Although the oracle bone language system is very complex, the auto-completions can be done in some simple ways. In fact, to begin with, we only need to create a text file such as .txt, .json, or csv, import any type of oracle bone phrases, such as the sixty stems and branches 甲子、乙丑、丙寅、丁卯、戊辰, to be used as a JavaScript variable, and then perform the following simple processing to achieve simple completion.

Auto-completion.js
// can also load text data from .txt、.csv files
const wordTriggers = ['甲子', '乙丑', '丙寅', '丁卯', '戊辰', '己巳', '庚午', ..., '壬戌', '癸亥']
monaco.languages.registerCompletionItemProvider('jiaguwen', {
 provideCompletionItems: function (model, position) {
   // range variable below defines the range of characters to be matched and parsed. A single character, sentence or an entire paragraph can be used as the target-to-be-matched. But here, we only need the last typed character before the cursor position
   const range = {
     startLineNumber: position.lineNumber,
     endLineNumber: position.lineNumber,
     startColumn: Math.max(1, position.column - 1),
     endColumn: position.column,
   }
   const lastChar = model.getValueInRange(range)
  // load the pre-defined auto-completion items, then use a list loop to filter out any entries that contain the last typed character. For example, if user's last typed character is "癸", then any entries such as "癸亥", "癸酉" that contain "癸" will be filtered and returned as prompts
   const wordSuggestions = wordTriggers
     .filter((str) => str.includes(lastChar)) // if contain targeted character
     .map((item) => {
       return {
         filterText: lastChar, // text to be matched
         label: item, // the label hint for the item in the dropdown menu
         kind: monaco.languages.CompletionItemKind.Text,
         insertText: item, // the text to be inserted or substituted
         detail: '辞典词条', // the type/detail hint for the entry in the dropdown menu
         range: range,
       }
     })
   return {
     suggestions: wordSuggestions,
   }
 },
})

After the simple code implementation above, we can achieve the following result:

辞条补全

It is apparent that the above code only uses the most basic processing logic, i.e. using a one-dimensional array, traversing each word in the list, if the current word contains the last character before the cursor, then it is considered a match, and then the word is returned as a auto-completion suggestion in a data format that would be accepted by monaco (ICompletionItem). But even with the basic logic, if we have a large enough data set, such as by collecting all common oracle bone vocabulary and sentences like from the book 甲骨文词谱 (Oracle Bone Vocabulary Compilation), 殷墟甲骨学大辞典 (The Great Dictionary for Yinxu Oracle Bones), then even in this primitive programming state, the text completions can also achieve fairly impressive results, which would be better than most traditional input tools.

Naturally, one-dimensional arrays are only suitable for cases where there are few entries. In cases where there are many entries, we would like to provide users with more entry prompts (such as using the detail property to indicate the current entry's type as "stem-branch", "person name", "sacrificial rite name", etc.), or we need to provide more filtering conditions based on the text context to avoid returning too many irrelevant results and overload the auto-completion list. Furthermore, we may also want to provide more text-editing operations after inserting the replacement text, such as grammar checking, text formatting, AI completion, etc., which can also be done by relevant code logic setup. In addition, we may also want to improve the editor's computational performance to prevent computing delays and lags, and we can consider using more advanced data structures such as trie trees or dictionary trees for more efficient matching.

Activating alternate typing modes with special characters as triggers: If slightly more complex operations are to be performed, such as using @, % to trigger different input modes like the current editor example on this page, it is also feasible. In practice, this kind of processing is also very necessary, as we would want to divide different functions and delegate them to separate processing logic, rather than to mix them all in a single function where we frantically try to untangle the conditions special to each scenerio - an essential code refactoring technique. Doing this will avoid returning too many suggestions, which are not conducive to data maintenance and updates. At the same time, different auto-completion types also would require different data types. For example, Strict{Loose}-style completions cannot be done by just using one-dimensional arrays like above, we need additional tags to tell us which are loose-style, which are strict-style, which are pinyin, etc. For example, we can use the dict (python) or Object (js) data types instead:

Dictionary.js
{
  严式: '𢦏', // strict form
  宽式: '', // loose traditional form
  简体: '', // loose simplified form
  拼音: 'zai', // pinyin
  替代写法: '𢦏{災}', // substituted text
  类型: '名词', // grammatical type
}

In this case, when computing whether the character would match against this dictionary object, we can check if the character or the pinyin substring is contained in the object, and if so, returning the text in Strict{Loose} or Pinyin format will do, or other customized text formats can be used.

However, this is the most ideal method of data storage, and we can further define more properties such as 释文举例, 著录来源, etc if one desires. But in terms of practicality, we may find that such a refined data structure seems to be an overkill, many auto-completion methods in this dictionary can be replaced by other more efficient functions or tools (such as hyperlinking and navigating to other website pages), in which case the overloading details can be avoided. In some scenerios, we would also want to save memory space as much as possible, as the Monaco library source code and its built-in web workers already occupy ~60MB-90MB+ memory space, so it is not advisable to load too much data. The editor itself is just a language processor, designed to help with text input and writing. So balance has to striked between the complexity of the data structure and the actual needs of the application.

And if it is only required that a pinyin input, simplified-traditional conversion, or alternate form input method are implemented, we do not need complex data structures (for example there is no need to use objects or dictionaries to mark the type of each data entry), as our matching logic is very simple, as long as a list contains an item containing that certain character, then all the data items in that entry is considered relevant. So for the three functions mentioned above, they can even be implemented using the exact same logic. Taking the character "隻" as an example, we can define as follows to include all the alternative writing methods and pinyin related to "隻", so that when "隻" is detected in this list of items, all the other items in the list will be returned as suggestions:

['㸕', '獲', '隻', '获', '只', 'zhi', 'huo', 'jue']

In the provideCompletionItems function in Monaco, we can use the following logic to implement the feature:

auto-completions.js
const triggers = [['', '', '', '', '', 'zhi', 'huo', 'jue'], ["","𢦏", "𢦔", "", "zai"], ...]
monaco.languages.registerCompletionItemProvider('jiaguwen', {
 triggerCharacters: ['@'],
 provideCompletionItems: function (model, position) {
   // @ trigger logic
   let alternateSuggestions = []
   const lineUntilPosition = model.getValueInRange({
     startLineNumber: position.lineNumber,
     startColumn: 1,
     endLineNumber: position.lineNumber,
     endColumn: position.column,
   })
  //  only text after @ is considered and matched, any before @ is ignored
   let regex = /@[^0-9]+$/
   let match = lineUntilPosition.match(regex)
   if (match) {
     const matchedText = match[0].substring(1)
     const range = {
       startLineNumber: position.lineNumber,
       endLineNumber: position.lineNumber,
       startColumn: position.column - matchedText.length - 1,
       endColumn: position.column,
     }
     let alternateSuggestionsArray = triggers.filter((item) =>
       item.includes(matchedText),
     )
     for (let i = 0; i < alternateSuggestionsArray.length; i++) {
       let arr = alternateSuggestionsArray[i]
       for (let j = 0; j < arr.length; j++) {
         if (arr[j] === matchedText) continue
         if (arr[j].match(/[a-zA-Z]/) && arr[j].length > 1) continue
         const entry = {
           filterText: '@' + matchedText,
           label: arr[j],
           kind: monaco.languages.CompletionItemKind.Text,
           insertText: arr[j],
           detail: '替代写法',
           range: range,
         }
         if (alternateSuggestions.find((item) => item.label === entry.label))
           continue
         alternateSuggestions.push(entry)
       }
     }
     return {
       suggestions: [...alternateSuggestions],
     }
   }
}}

Namely, this is to traverse each array in the array, if the array contains the current text, then it is considered a match, and then all the other items of text in that array is returned as a prompt. This way, we can avoid overly complex code logic and data structures, and only need to traverse array in array, if there is one contains the current text, then it is considered a match, and then only the array containing the matched text is traversed to return each text (or entry) as a prompt, which is only of N+1 complexity (N+1 or less operations for number of N arrays). In the end, we can achieve the following result:

替代写法

Under this processing logic, no matter if it is @隻, @獲, @只, @zhi, @huo, any of these characters can trigger any other alternate writing methods, so we avoid overly complex code logic and data structures, and can achieve a more versatile input mode. In addition, in terms of data preprocessing and generation, we do not need to annotate all possible text pinyin and simplified-traditional combinations one by one, as there are more than 4000+ oracle bone characters, and annotating each one in detail of their spelling and alternate forms would be a great waste of human annotation resources. If the relationship between the oracle bone inscriptions changes in the future, we will have to make individual one-to-one modifications, which further complicates the logic and is not conducive to data integrity and maintainability. In fact, in the data preparation process, we can just use python to automatically convert between simplified and traditional characters and generate pinyin. Libraries in the python ecosystem such as OpenCC and pypinyin can achieve this very well:

Traditional-Simplified Conversion.py
import opencc
from pypinyin import pinyin, Style, lazy_pinyin

simple2trad = opencc.OpenCC("s2t.json") # Simplified to Traditional
trad2simple = opencc.OpenCC("t2s.json") # Traditional to Simplified

simple2trad.convert("") # Return "隻"
trad2simple.convert("") # Return "只"

lazy_pinyin("") # return "zhi"
pinyin("", style=Style.TONE3) # return "zhi3"

In this scenario, we only need a single standardization table, and then use a python script to automatically generate all the combinations of simplified-traditional and pinyin. This way, we only have to define the relationships between strict and loose standardization forms of a character. For example, a table of standardization can be defined as follows:

Archaic FormStandardizationStandardization Type
...Strict
...Loose
...Strict
...Loose

In doing so, we can greatly cut down on the amount of tedious work. When encountering some non-CJK characters, we can add some auxiliary data to a separate table. This way, we can achieve the maximum degree of data automation, reduce a lot of manual annotation work, and avoid overly complex code logic. If we want to complete the strict-loose correspondences like 鼎{貞}, we can also define the completionItem in a similar way, and the choice of data types can be varied:

Strict-Loose Definition.js
[
  {
    strict: [''],
    loose: [''],
  },
  {
    strict: ['𤑔', '', '', '𤆰'],
    loose: [''],
  }
]

// ------ or -------

[
  [[''], ['']],
  [['𤑔', '', '', '𤆰'], ['']],
]

Namely, we can use an array of objects or a two-dimensional array to define the strict-loose correspondences. And if we want to also auto-complete the component combinations, we can define the data type as follows:

component completions.js
const components = [
  {
: [['', '']],
  },
  {
    '𣈉': [
      ['', '', ''],
      ['', '', '', ''],
    ],
  },
]

Since each character above could have multiple component combinations, we can use a two-dimensional array to define them, which can provide more auto-completion choices. So, to reproduce the three major input modes in the current IDE editor on this page, a complete code example can be written as follows:

完整代码.js
const alternateTriggers = [
  ['', '', '', '', '', 'zhi', 'huo', 'jue'],
  ['', '𢦏', '𢦔', '', 'zai'],
]
const componentTriggers = [
  {
: [['', '']],
  },
  {
    '𣈉': [
      ['', '', ''],
      ['', '', '', ''],
    ],
  },
]
const looseStrictTriggers = [
  [[''], ['']],
  [['𤑔', '', '', '𤆰'], ['']],
]
const wordTriggers = [
  '甲子',
  '乙丑',
  '丙寅',
  '丁卯',
  '戊辰',
  '己巳',
  '庚午',
  '壬戌',
  '癸亥',
]

monaco.languages.registerCompletionItemProvider('jiaguwen', {
  triggerCharacters: ['@', '%'], // two special trigger characters, can also be any other characters
  provideCompletionItems: function (model, position) {
    // @ substitution mode
    let alternateSuggestions = []
    const lineUntilPosition = model.getValueInRange({
      startLineNumber: position.lineNumber,
      startColumn: 1,
      endLineNumber: position.lineNumber,
      endColumn: position.column,
    })
    let regex = /@[^0-9]+$/
    let match = lineUntilPosition.match(regex)
    if (match) {
      const matchedText = match[0].substring(1)
      const range = {
        startLineNumber: position.lineNumber,
        endLineNumber: position.lineNumber,
        startColumn: position.column - matchedText.length - 1,
        endColumn: position.column,
      }
      let alternateSuggestionsArray = alternateTriggers.filter((item) =>
        item.includes(matchedText),
      )
      for (let i = 0; i < alternateSuggestionsArray.length; i++) {
        let arr = alternateSuggestionsArray[i]
        for (let j = 0; j < arr.length; j++) {
          if (arr[j] === matchedText) continue
          if (arr[j].match(/[a-zA-Z]/) && arr[j].length > 1) continue
          const entry = {
            filterText: '@' + matchedText,
            label: arr[j],
            kind: monaco.languages.CompletionItemKind.Text,
            insertText: arr[j],
            detail: '替代写法',
            range: range,
          }
          if (alternateSuggestions.find((item) => item.label === entry.label))
            continue
          alternateSuggestions.push(entry)
        }
      }
      return {
        suggestions: [...alternateSuggestions],
      }
    }
    // % division mode
    const componentSuggestions = []
    regex = /%[^0-9]+$/
    match = lineUntilPosition.match(regex)
    if (match) {
      const matchedText = match[0].substring(1)
      const range = {
        startLineNumber: position.lineNumber,
        endLineNumber: position.lineNumber,
        startColumn: position.column - matchedText.length - 1,
        endColumn: position.column,
      }
      // here we define a temporary function to determine if an array is a subset of another array
      function isSubset(subset, main) {
        const mainFreq = main.reduce((acc, val) => {
          acc[val] = (acc[val] || 0) + 1
          return acc
        }, {})
        for (const element of subset) {
          if (!mainFreq[element]) {
            return false
          }
          mainFreq[element]--
        }
        return true
      }
      const textSplit = matchedText.split('')
      let componentSuggestionsArray = componentTriggers.filter(
        (item) =>
          Object.values(item)[0].filter((arr) => isSubset(textSplit, arr))
            .length > 0,
      )
      for (let i = 0; i < componentSuggestionsArray.length; i++) {
        let obj = componentSuggestionsArray[i]
        const key = Object.keys(obj)[0]
        const entry = {
          filterText: '%' + matchedText,
          label: key,
          kind: monaco.languages.CompletionItemKind.Text,
          insertText: key,
          detail: '部件构字',
          range: range,
        }
        if (componentSuggestions.find((item) => item.label === entry.label))
          continue
        componentSuggestions.push(entry)
      }
      return {
        suggestions: [...componentSuggestions],
      }
    }

    // the default typing mode, i.e. no @ or % is used as triggers
    let finalPairSuggestions = []
    let finalWordSuggestions = []
    // here we need to consider the possibility of `surrogate pair`, so we need to locate the two character positions before the cursor, and then perform a reverse character split
    for (let i = 1; i <= 2; i++) {
      const range = {
        startLineNumber: position.lineNumber,
        endLineNumber: position.lineNumber,
        startColumn: Math.max(1, position.column - i),
        endColumn: position.column,
      }
      const lastChar = model.getValueInRange(range)
      let pairSuggestions = looseStrictTriggers.filter(
        (twoSegArr) =>
          twoSegArr[0].includes(lastChar) || twoSegArr[1].includes(lastChar),
      )
      if (pairSuggestions.length > 0) {
        let suggests = []
        for (let i = 0; i < pairSuggestions.length; i++) {
          let pair = pairSuggestions[i]
          const stricts = pair[0]
          const looses = pair[1]
          // map each strict-loose pair
          for (let j = 0; j < stricts.length; j++) {
            for (let k = 0; k < looses.length; k++) {
              const entry = {
                filterText: lastChar,
                label: `${stricts[j]}{${looses[k]}}`,
                kind: monaco.languages.CompletionItemKind.Text,
                insertText: `${stricts[j]}{${looses[k]}}`,
                detail: '宽严式对应',
                range: range,
              }
              if (
                finalPairSuggestions.find((item) => item.label === entry.label)
              )
                continue
              suggests.push(entry)
            }
          }
        }
        finalPairSuggestions.push(...suggests)
      }
      // auto-completion of oracle bone vocabulary
      const wordSuggestions = wordTriggers
        .filter((str) => str.includes(lastChar))
        .map((item) => {
          return {
            filterText: lastChar,
            label: item,
            kind: monaco.languages.CompletionItemKind.Text,
            insertText: item,
            detail: '辞典词条',
            range: range,
          }
        })
      finalWordSuggestions.push(...wordSuggestions)
    }
    // remove duplicates or sort data entries (optional)
    finalPairSuggestions = Array.from(
      new Set(finalPairSuggestions.map((a) => a.label)),
    ).map((label) => {
      return finalPairSuggestions.find((a) => a.label === label)
    })
    finalWordSuggestions = Array.from(
      new Set(finalWordSuggestions.map((a) => a.label)),
    ).map((label) => {
      return finalWordSuggestions.find((a) => a.label === label)
    })
    return {
      suggestions: [...finalPairSuggestions, ...finalWordSuggestions],
    }
  },
})

The complete code above is also copied as a demo in monaco-playground editor. One can refer here if one is interested in the implementation details.

The aforementioned auto-completion methods can basically cover for the most of the needs of oracle bone inscriptions, i.e. the find character and complete character functions. Counting the feature of "common word completions", the typing and editing efficiency of the oracle bone inscriptions can be greatly improved. In addition, in the future development, we can also consider the inclusion of functionalities like code snippets to further enhance the user experience (especially transcription snippets and transcription templates), but this is based on the premise of having a complete transcription database. As of now, this is not yet possible.

Secondly, besides the two types of syntax highlighting and word completion functions, we may also be interested in the hover tooltip feature of monaco. This feature triggers a callback function by observing the mouse hover effect in the editor interface. By customizing the processing method, we can achieve data queries and quick access additional details without leaving the page. For example, when the user hovers over a text, we can display more detailed information like phonology, typological classification, etc. of the character based on the hovered content. This can help users better understand and use the character without having to go to the dedicated character detail page, saving some additional operations. This feature is also supported in monaco. To implement, just define a custom processing function in registerHoverProvider(), and then use the relevant api to navigate to the database.

For an example of implementation, we can first define a data query function. Here, we use two commonly used libraries in the vue.js framework, but any other type of data fetching logic can achieve similar effects, such as the fetch() function in js and lodash's _.debounce() etc.:

api query.js
import axios from 'axios'
import { useDebounceFn } from '@vueuse/core'
const debouncedServerLookUp = useDebounceFn((text) => {
  return axios
    .get('/fetch-data-from-server/', {
      params: {
        name: text,
      },
    })
    .then(async (res) => {
      return res?.data ?? {}
    })
    .catch((err) => {
      throw new Error(err)
    })
}, 3000)

The relevant data queries naturally also require a backend and a sql database to support. To fully implement this feature, one needs to have a good understanding of full-stack web development. Here, we only simulate a basic data query process. We use the axios library to send a GET request to the backend, and use useDebounceFn to limit the frequency of requests, so as to avoid excessive server load caused by frequent requests when the user moves the mouse quickly.

Next, we integrate the data query logic into the relevant callback function. When the user hovers over a text, the API endpoint for data access is triggered, then the data is queried, and finally returned:

data processing.js
monaco.languages.registerHoverProvider('jiaguwen', {
  provideHover: function (model, position) {
    const range = new monaco.Range(
      position.lineNumber,
      position.column,
      position.lineNumber,
      position.column + 1,
    )
    const text = model.getValueInRange(range)
    // if is number, letter or special characters, no need to query at all
    if (text.match(/[a-zA-Z0-9.,。:;“”‘’??!!\-\=\[\]\{\}\\\|]/)) return
    // database query
    return debouncedServerLookUp(text)
      .then((res) => {
        return {
          range: new monaco.Range(
            position.lineNumber,
            position.column,
            position.lineNumber,
            position.column + 1,
          ),
          contents: [
            {
              value: `# 字形信息 - ${text}`,
            },
            {
              value: `### 字库链接: [link](${fill_url_here}/glyphs/${res.字头})`,
            },
            {
              value: `### 严式隶定: ${res.严式隶定 ?? ''}`,
            },
            {
              value: `### 宽式隶定: ${res.宽式隶定 ?? ''}`,
            },
            {
              value: `### 隶定拼音: ${res.隶定拼音 ?? ''}`,
            },
            // ... other information
          ],
        }
      })
      .catch((err) => {
        return
      })
  },
})

The above code is a simplified version of the hover tooltip implementation. And the programming difficulty of this is not particularly high either. And in terms of functionality, it is not as unique as the word completion feature. In general, even if you leave the Monaco editor environment, any ordinary web page on the modern web can observe the user's mouse clicks, movements, and hover effects, so as long as we want to implement something like this, we can integrate similar features into any web page. Therefore, it is only used as an auxiliary function here. What we are more concerned about is the overall processing idea after the keyboard and mouse events, that is, what information to provide to improve the user's input efficiency and experience. One way to go about it is that in the future, when the data volume increases, we can use a specially trained AI language model to analyze the text that the user hovers over or select, and then provide more intelligent paragraph explanations, which is actually a real-time AI translation function.

Lastly, Monaco also provides some other types of functions. In the development process of the current IDE module, due to the lack of relevant data or application scenarios, they have not received much attention and concentrated development. However, with the accumulation of data and technological innovation in the field of ancient texts, they may also have certain utilities in the future. Here is a brief discussion:

  • Diff Editor: A dual-text parallel editor. That is, two editors are set on the same page, and after entering similar content, the text lines that are inconsistent are highlighted, such as the differences between the two versions in terms of text addition, deletion, and modification. As shown below:Diff Editor
    This kind of application scenario in ancient texts is most likely to be used in comparing two versions of a text, where you can visually see the differences between the two versions. For example, when making a text of a certain record, you can directly put the text content of the previous 合集 or 摹系 compliations into the editor, and then compare the differences between the new and old versions, so as to better proofread and modify. But this requires each record to have a complete text database, and use the same characters/fonts for display, in order to have the possibility of application. Judging from the current situation of the "thumbnail pasting" of various texts, the application scope of this function seems to be limited.
  • Inline Word Hints: - which is using a gray text annotation for a word in the editor. This is mainly used in programming languages to annotate general variable types and parameter types, and the functions can provide syntax and parameter information in real time according to the backend documentation. This seems to have some application scenarios in ancient texts. Especially the display of syntactic components, such as the "pretext", "occupancy", and "verification" of oracle bones, can be generated in real time according to the analysis of the paragraph content, as shown below:Inline Word Hints
    This feature can help analyze the grammatical structure of ancient texts. However, the implementation of this feature requires the establishment of a complete syntax analysis logic and system, and the construction of such a system also requires a deep understanding and research of ancient texts, as well as sufficient material support. Moreover, from the perspective of natural language, it seems that syntax analysis models based on AI will be more effective than any complex traditional algorithms.
  • Symbol Search and Highlighting: This feature is quite similar to the "inline word hints" mentioned above, which also involves the parsing and annotation of syntactic components at the symbol level, but the difference is that the former is comment-based, while the latter can provide very powerful query and highlighting effects. For example, given a long text in the editor, we can define each word or character according to predefined tags or analysis algorithms, such as determining which are person names, clan names, place names, and which are subject, predicate, and object, and users can highlight all similar syntactic components according to the symbols, as shown below:Symbol Search
    The so-called symbolic level search is not based on specific words, but on the search for similar things. For example, person names, then there will be 殻、賓、爭、扶 and other diviner names, which usually appear in the form of "殻貞", "賓貞", and so on, not limited to a single word and sentence structure. Similarly, the sequence of "one", "two", "three", "four" appears in the text in the form of "one.", "two.", and so on, and there is more than one number. So we can define tags for similar content. Usually, similar functions seem to be common in a pre-annotated database, where we invite experts and scholars to annotate every word and sentence in the database. But the difference is that in the monaco editor environment, its functions can be calculated in real time, so it can be applied to the analysis of custom text and datasets, but this also requires us to do automated programming analysis of language materials. But relatively speaking, if it is only basic character-level annotation, then through the systematic annotation of the Jingyuan Oracle Bone Character Database (such as proper noun annotation), we can already achieve a very powerful symbolic analysis function. And this will also be one of the main tasks to be completed in the future. Some other small functions, such as syntax error highlighting, text folding, text formatting, etc., are widely used in various programming languages to varying degrees, but are not very applicable to the current academic field.

In summary, the above is a brief introduction to the use of monaco-editor in the field of oracle bone inscriptions. The current implementation is based on the existing data and technology, and is only a temporary function. Given the serious lack of a unified oracle bone inscription database, vocabulary database, character database, text annotation database, and font, we can only fill in some technical gaps through some data collection and organization based on personal efforts. The data collection during the production period is not easy, and it is inevitable that it will be subject to personal knowledge and ability limitations, and there will definitely be many unreasonable processing methods. This will require continuous improvement and updates in the future.