In Part 1, I introduced the evolution of integration testing for Nebula Graph. Now we will add a test case into the test set and run all the test cases …<\/b>
\nJoin the DZone community and get the full member experience. In BDD-Based Integration Testing Framework for Nebula Graph: Part 1, I introduced the evolution of integration testing for Nebula Graph. In this article, I will introduce how to add a test case into the test set and run all the test cases successfully. At the beginning of building the testing framework for Nebula Graph 2.0, we developed some tool classes to help the testing framework quickly start and stop a single-node Nebula Graph cluster, including checking for port conflicts and modifying part configurations. Here is the original execution procedure: However, parameters need to be passed transparently to the pytest.main function to specify some parameters of pytest, and the scripts generated by cmake are needed to execute a single test case, which makes the framework not so convenient for users. What we want to achieve is executing a test case where it is located. During this improvement of the testing framework, besides the changes to the program entry, most of the original encapsulated logic is reused. A lot of test cases have been accumulated for Nebula Graph, so single-process operation can no longer meet the requirements of fast iteration. We have tried several parallel test executor plugins, considering compatibility requirements, we finally chose pytest-xdist to accelerate the testing procedure. Pytest supports fixtures across these five scopes: session, module, class, package, and function. However, we need a global fixture to start and initialize the Nebula Graph services. Currently, for a session-scoped fixture, the highest level, each runner needs to be executed once. For example, if there are eight runners, eight Nebula Graph database services must be started, which is not what we want. According to the documentation of pytest-xdist, a lock file is needed for inter-process communication between runners. To make sure that the control logic is simple enough, we separate the logic for starting and stopping the program and for preparation from the process of executing the test, that is, a single step is used to start Nebula Graph, and when errors occur to some tests, Nebula Console is connected to the Nebula Graph database that is in the process of testing for validation and debugging. Before the new framework, to import data to Nebula Graph, an entire INSERT statement in nGQL is executed, which causes the following problems: To solve these problems, referring to the implementation of Nebula Importer, we separate the importing logic from the dataset completely and implement a new importing module in Python. However, so far, only CSV files are supported and one CSV file can store only one tag or edge type data. According to the new importing module, the structure of the dataset for Nebula Graph testing becomes clear. Each directory has all the CSV files for one graph space. The description of each file and the details of a graph space are configured in config.yaml in each directory. In the preceding example, the two graph spaces, ‘ basketballplayer ‘ and ‘ basketballplayer_int_vid ‘, share the same dataset. To add a new dataset, only a directory like ‘ basketballplayer ‘ is needed. For more information about config.yaml, see the nebula-graph repository. Besides pytest and nebula-python, the commonly used libraries, some plugins such as pytest-bdd and pytest-xdist are used in the testing framework. In addition, to better unify the format of the FEATURE files for adding test cases, reformat-gherkin is introduced and some format modifications are made to align the format with that of openCypher TCK FEATURE files. Currently, nebula-python and reformat-gherkin are installed with their source code. To simplify the installation, a makefile is provided in the nebula-graph\/tests directory. To prepare the environment for the testing, run the following command. The format check procedure has been added to the GitHub Action CI process. If your files are not compliant with the expected format, run make fmt to format them. According to Part I, the BDD-based testing framework for Nebula Graph is a black box testing process, which means you do not need to know how your statements are called or which function to call is more in line with the expected result. What you only need to do is writing a FEATURE file in natural language based on the rules. Here is a test case example. In a FEATURE file, the Given section is for the initial conditions for the test. In this example, it creates a graph with space named «basketballplayer». The When section contains the inputs for testing, that is, nGQL statements. In this example, the Then section gives the expected results and the expected comparing method, that is, the records of the table should be compared in a relaxed and unordered manner. The FEATURE files are written in the Gherkin language. A FEATURE file is composed of the following sections: Each scenario has its own steps. Each step is composed of these: According to the preceding description, each Scenario is composed of several steps. In Nebula Graph testing framework, its format is compliant with the rules of openCypher TCK and some special steps are developed to simplify editing of the test cases: Besides the preceding steps, more steps can be defined to speed up the development of test cases. openCypher TCK defines the format of the expected results. The format of vertices and edges is borrowed from the pattern of MATCH, so if you are familiar with openCypher query language, you can easily understand the results of the TCK test cases. For example, the format of part graph semantics is as follows: However, Nebula Graph differs a little from Neo4j in their graph models. For example, in Nebula Graph, each tag can have its own properties, so according to the existing openCypher TCK rules, a vertex with multiple tags that have their own properties cannot be described. The description of edges has a similar problem. In Nebula Graph, an edge key is of four-tuples , but the existing openCypher TCK rules do not support src, dst, and rank. Therefore, to solve these problems, we expand the expressions of the expected results: The expanded expressions of vertices and edges are compatible with the existing TCK test cases and fit the design of Nebula Graph. In addition to the expression problem, the next one we faced was how to efficiently and accurately convert the expression into a specific data structure so that it can be compared with the actual query results. After considering solutions such as regular expression matching and parsing by a parser, we decided to construct a parser to process the strings with specific syntax rules. This solution has the following advantages: With ply.yacc and ply.lex, we can use a small number of code lines to implement the complex requirements described above. For more information about the implementation, see nbv.py. Currently, the testing procedure is as follows: 1. Edit the FEATURE files. Currently, all the FEATURE files for Nebula Graph can be found in the tests\/tck\/features directory of the github.com\/vesoft-inc\/nebula-graph repository. 2. Start the Nebula Graph services. 3. Execute the testing locally. 4. Stop the Nebula Graph services. When the test cases need to be debugged, you can use some methods supported by pytest to debug them further. For example, run these commands to execute the test cases that failed in the last process. Alternatively, you can add a mark to a specific scenario in the FEATURE file and execute the marked case only. For example, run these commands: Standing on the shoulders of our predecessors allows us to find a more suitable test solution for Nebula Graph, so we would like to thank all the open-source projects mentioned in this article. In the process of practicing pytest-bdd, we have found some imperfections. For example, it has compatibility issues with plugins such as pytest-xdist (gherkin-reporter), and pytest does not natively provide fixtures of the global scope level. However, in general, the benefits it brings to Nebula Graph far outweigh these problems. In Part 1, I mentioned that the new testing framework enables no-programming. It is not a fantasy. When the mentioned mode is fixed, we can develop a scaffold for adding test cases, allowing users to ‘fill in the blanks’ with data on pages to automatically generate corresponding FEATURE files, which can further facilitate users. If you are interested, welcome to contribute to this idea. Published at DZone with permission of Xinglu Yee. See the original article here. Opinions expressed by DZone contributors are their own.<\/p>\n