Tests

1. Tests in the application can be standalone or built-in In Course, training program, group (for Bitrix 24):

• Independent - are displayed in the test catalog, in the personal account - in the My Tests section.
• Built-in:
  • Built-in CourseIt is displayed in the course catalog and in the My Courses section within the course.
  • Built-in programIt is displayed in the program catalog and in the My Programs section within the program.
  • For Bitrix24. If Test is added to Bitrix groupthen it will be displayed to the user in the My Groups tab, within the group. In the catalog Test, it will be available only to members of this group.

2. VisibilityTests can have different visibility settings (specified during creation):

• Hide in catalog. Test will be hidden in the catalog, but if assigned, it will be available in the My Tests section.
• For units only. Test will be available to employees of selected units.
• Hide from extranet. For Bitrix 24. Test will be hidden from Bitrix 24 portal extranet users.

Availability Period. When this setting is activated, Test will be available in the catalog during the selected period.

3. Methods of addingTest can be added via the interface or by uploading an Excel file. The template for uploading can be downloaded by clicking on the Import element - Download sample xlsx file for import.

1. Key capabilities of the Nika.AI assistant in testing:

• Creating a test from scratch, based on a free-form request;

• Refactoring existing tests into the LMS test structure;

• Creating tests based on uploaded materials.

Creating a test from scratch. Enter the topic, test name in the "Theme test" field, and Nika.AI will generate Test based on open data.

Reprocessingimport. If you have a ready-made Test in Word or Excel, you can use Nika.AI as a test import tool. Upload your Test in the "Theme test" field, and Nika will analyze it, process it, and present it in the Brusnika.LMS test template. 

Document-based creation. You can upload any document in Word or Excel format and write in the Theme field a task to create a test based on the uploaded document: Nika.AI will analyze the document, generate Test based on it, and present it in the Bursniki.LMS test template. 

It is recommended to upload documents not more than 5 MB andmore than 300,000 characters.

Large files:

• increase the processing time;

• do not guarantee the correctness of the analysis;

• reduce the remaining tokens.

2. Automatic phrasing adaptationNika.AI automatically:

• corrects errors in the text;

• eliminates duplicate questions;

• replaces duplicate answer options.

3. Quality of generated tests

Nika.AI uses a probabilistic text generation model. Therefore, every Test created, even from the same query, may differ.

4. Number of tokens per portalEach LMS system is allocated a free monthly limit of 10 thousand tokens.

Estimated number of tokens consumed:

• One Test with 10 multiple-choice questions = approximately 2 thousand tokens. Without analysis of uploaded documents.

• Analysis of one uploaded document with a volume of ~15,000 characters (approximately 25 pages in 11pt Arial font) will require approximately ~2,000-3,000 tokens.

The limit resets monthly. Accumulation of free tokens does not occur.

5. Acquisition of tokens. Additional tokens can be purchased in the section My Plan - Nika.AI tokens.

6. Personal dataNika.AI does not store personal data.

7. Number of questions in the created test cannot be more than 20If you need a Test with a large number of questions, create several tests and combine them using Excel export: create Test - publish in the test catalog - export to Excel; the second Test in a similar manner; then combine both tests in Excel and add them to Catalog via Test Import.

1. Nika.AI Interface in the tests, it represents a Dialog Box with additional clarifying parameters:

• Number of questions;

• Number of responses;

• Pass-through Score;

• Question type;

• Difficulty level.

As well as control elements:

• Preview;

• Add to Catalog;

• Clear.

2. Token limitAt the top of the dialog window, the token limit is displayed - how many have been used and how many remain.

3. Theme test, attachment of materials. In the Theme test field, you can enter the test name and add materials that need to be used for creating the test.

Theme can be entered in any form, but the better the Theme, query, is specified, the more accurate and higher quality the resulting text will be.

You can upload documents in Word and Excel formats to create tests.

Recommendation for writing a test creation query

If a general wording is used:

"Make Test on computer work"

AI is forced to guess the direction:

• Windows?

• Linux?

• BIOS?

• Safety?

• office software?

• networks?

This may lead to unwanted displacement.

Therefore, it is recommended to formulate the query as detailed as possible:

"Create a Test on PowerShell for beginners with emphasis on file operations and loops."

4. Number of questions. Specify the number of questions for the test, and Nika.AI will create Test with the specified number of questions.

5. Number of responses. How many responses to generate per question. For questions of type Choice, Matching, Sequence, the specified number of answer variants or options will be generated.

6. Pass-through Score. This parameter does not affect the prompt and test generation results, but is required as a mandatory option for adding the test to Catalog.

7. Question type. The types of questions to be added to the created Test are selected. All question types are active in By Default.

8. Difficulty level. It affects the prompt, and Nika.AI will create Test of the corresponding difficulty level depending on the choice. Of course, a parameter such as "difficulty level" is quite subjective, nevertheless.

9. After creating a test, a slider "Publish immediately. It By Default is active and is used when publishing the created test in the catalog. If the slider is in the "Publish Immediately" position, then Test will be immediately published in the catalog; if the slider is turned off, then Test will be published in the Draft status.

10. Preview shows or disables test editor results on the right.

11. Add to Catalog. When clicked, the created Test will be added to the Catalog of tests, and the Dialog Box with Nika.AI will be closed.

12. Clears the form.

1. The result in the form of a test is presented in preview mode. In the created test, you can:

• Swap questions around;

• Edit questions and answers;

• Add answers to questions;

• At the bottom of the test draft, create new questions.

1. To add a Test, open Catalog tests. Click the plus "Add". The "Add" element is available only to LMS moderators.

2. In test settings mandatory there are three fields (in addition to the need for the questions themselves):

• Test Name (do not use special characters in names, such as the snowboarder emoji).
• Time on Test.
• Pass-through Score.

3. Field "Competence" and function "Generate IDP when failing to achieve the passing score".

Add competencies to the test with and without activating the "Generate IDP if passing score is not achieved" element.

Without activation. In this case, the competencies will serve as information only and will not provide any additional functionality.

Upon activation. Activation will only be possible if developmental activities are assigned to the competency in the Settings - Competency Management section. If developmental activities are assigned but the user has not achieved the passing score Score, the system will automatically generate IDP, which will already include the developmental activities.

4. Enable comments. When activated, all users will be able to write comments on the test card.

5. Issue Certificate. When activating, you must select Certificate. Certificate is created in the Settings - Application Settings - Certificates section. Upon successful completion of the test, the system will generate Certificate, which will be available to the test taker in the My Certificates section.

6. Feedback Form. Анкета создается в разделе Настройки - Настройки приложения - Feedback Form. При прохождении теста система откроет анкету тестируемому для заполнения. Результаты заполнения анкет можно увидеть в разделе Настройки Отчеты - Отчет Анкеты обратной связи.

7. Points. How many points to award for passing the test. This is not a passing Score, but part of the gamification system. 

8. Passing score. Calculated as a percentage, taking into account the weights of the questions if they are provided. 

9. Test Results. Four options are available for selection: 

• Do not show answers. Test results will be available only to the application moderators.
• display questions with scoring. The employee taking the test will see Correct or Incorrect given Answer. At the same time, they will not see the correct results (detailed breakdown). 
• display questions, answers, score. The employee taking the test will see Correct or Incorrect given Answer. They will also see which answers contain errors and which answers are correct.
• display Incorrect Answer immediately. The employee, when taking the test, will immediately see whether they answered correctly or incorrectly (the result will be highlighted). However, they will not see the correct answers.

By Default works with the first option "do not show answers". 

In the second and third options, the employee can view the test results in their personal account under the My Tests section by clicking on the corresponding Test. The "Test Results" element will be available in the test. 

1. Introduction of questions and their answers takes place in the Test Questions tab. 

2. The following types of questions are available in the application:

• Choice - Choice one or more correct answers.
• Sequence - restoration of the correct sequence.
• Matching - restoration of correct correspondence.
• Open - Open question type. In it, you need to select a verifier who will check the answers and record the results.
• SubstitutionIt is necessary to fill in the blanks.

If you need to create a question with one correct answer and inform the user about it, then after the main text of the question, add the text: Select one Correct Answer.

If you need to create a question with answer options: Yes, No. When creating the answers, create the corresponding options: Yes, No.

3. When creating a question type Choice and if the "Show Answer immediately" option is selected in the test results, you can fill in the feedback that the system will provide for correct or incorrect answers:

4. Enter the question and its answers sequentially.

5. The weight of the matter. 

By Default All questions have the same weight equal to one. The weights are then recalculated based on the number of questions entered. That is, if there is one question in the test, its weight will be 1 or 100%. If there are two questions in the test, the weight of each will be: 0.5 or 1 / 2 * 100 = 50%. If there are 8 questions in the test, the weight of each will be: 

• The sum of weights of all questions = 1 + 1 + 1 + 1 + 1 + 1 + 1 + 1 = 8
• The weight of one question will be: 1 / 8 * 100 = 12.5%

Weight of specific question / Total sum of weights * 100

The application allows you to assign an arbitrary weight to any question. To do this, enter an integer in the special field, and the system will convert the weights of all questions into percentages using the formula above.

Example. Test of 8 questions. Let's assign a weight of 7 to the last question, 3 to the first question, and leave the rest blank. Let's calculate the question weights.

The sum of weights of all questions = 3 + 1 + 1 + 1 + 1 + 1 + 1 + 7 = 16

The weight of the first question will be:

3 / 16 * 100 = 18.75%

The weight of the last question will be:

7 / 16 * 100 = 43.75%

The weight of each of the remaining questions will be:

1 / 16 * 100 = 6.25%

Checking (should equal 100%): 18.75 + 6.25 + 6.25 + 6.25 + 6.25 + 6.25 + 6.25 + 43.75 = 100%

The pass score Score is calculated taking into account the weighting coefficients of all questions.

In the example above, if we set the passing threshold Score to 42%, let's consider different scenarios:

• The first seven questions are correct, the last one is incorrect, so the score will be 56.25%, Test Passed. 
• Last one correct, all others incorrect, scored 43.75%, Test Passed.

Note that percentages are rounded on the screen, but not in calculations and formulas.

1. Using the "Blocks" functionality, you can create thematic blocks and then activate or deactivate them depending on testing tasks.

For example, a database of 100 questions is created, consisting of 10 thematic blocks with 10 questions each. Subsequently, some blocks are activated while others are deactivated. The application will present the test-taker with questions only from the active blocks. 

2. This functionality is convenient in situations where it is necessary to create a large question database and then, depending on the tasks, conduct testing not on the entire list of topics, but on specific topics.

3. The general workflow for working with blocks is as follows:

1. Thematic blocks are activated. A tab for creating thematic blocks appears.
2. Thematic blocks are created in the "Blocks" tab. Only the name of the new block needs to be specified. 
3. When creating questions, each question is linked to a specific thematic block.
4. Next, when assigning the test, the required blocks are activated, and the unnecessary ones are deactivated.
5. The system will present the test taker with questions only from active blocks.

1. Tests in the application can:

• Create, edit - only by the application moderator.
• Copy, delete, import, export - application moderator only.
• Pass through - any employee.

2. Creating, editing a test. For creation test click on the plus sign add in the test catalog and sequentially fill in the tabs:

• Test setup;
• Test questions;
• Blocks (when activated).

For editing test press on Edit element in test card.

3. Copying, deleting, importing, exporting tests. 

For copying test click on the element copy in test cardTest will be copied and appear in the directory with the name Copy. Test will be in the Draft status.

For deletion test click on the element Delete in test cardTest will be permanently deleted.

Import testThe Test can be imported in the test directory by clicking on the Import element. The Test can be imported in two formats: UDATA and Excel. The format is selected in the application settings: Settings - Application Settings - Other - Export Format.

Import in format UDATAThis format is used for backing up tests or migrating them from one portal to another. 

Import in format ExcelConvenient to use when creating large question databases. Download the test template for import "Download sample xlsx file for import". Prepare your question database in it and import.

Export testTo export, click on the element export Hello! How can I assist you today? test cardTest will be exported in the format specified in the settings (UDATA or Excel).

Tests have the following statuses:

• Status publications: published; draft (unsaved, not published). 
• Passing statuses: Not Started; Passed; Not Passed.
• Statuses by purpose: Self-Enrollment - the employee clicked on Take the test, appointed - appointed as the application moderator.

Answer:

If the Publish button is not active, check the following options:

• One of the required fields (name, time on Test, passing Score) is not filled;
• Specific symbols have been entered in the test name (e.g., a snowboarder emoji);
• Questions not added;
• No correct answers selected;
• No reviewers selected for open questions (if any);
• In the test catalog, there is Test with the same name.

Answer:

Test falls into Section My tests in the following cases:

• The user started taking the Test test from the test catalog. Test will be added to Section Completed or In Progress attempts.
• Test is assigned to the user (not as part of a course or program, but as a separate Test).
• Test and the user have been added to the same group (for Bitrix 24).

If the user does not see Test in their personal account under My Tests, the following scenarios are possible:

• Not Startedabout Completion.
• Test is included in the course. Then it will be in the My Tests section.
• Test in the program, then it will be in the My Programs section.
• Test is included in the Group (for Bitrix 24). Then Test will be available in the My Groups section.

Answer:

If the user does not see Test in the test catalog, the following options are possible:

• Test in the course. Then Test will be in the course catalog.
• Test in the program. Then Test will be in the program directory.
• Test in the group (for Bitrix 24). Then Test will see the test catalog only the members of this group.
• Test is hidden in settings when creating a test: hidden in the catalog or available only to specific users.
• Availability Period test has expired. When creating a test, you can specify Availability Period after which Test will not be available in the catalog.

Answer:

Yes, you can add any number of questions to Test. There are no restrictions.

Answer:

Enter the answer option Yes, enter the answer option No and select Correct Answer.

Answer:

By Default all tests are open and accessible to all employees in the tests directory when published. Tests can be hidden or their availability restricted in the following ways.

When creating a test in test settings:

• Hide in the test catalog. Test no one will see it. If assigned, it will be visible to employees to whom it is assigned in the My Tests section.
• Restrict availability for specific departments (employees). In this case, Test will only be accessible to selected employees.
• Hide from extranet (for Bitrix 24). In this case, this Test will not be visible to extranet users.
• Availability Period. When creating a test, select its availability period. After the availability period ends, Test will be hidden in the catalog.
• Add to group (for Bitrix24). If Test is added to a group in the application (linked to a Bitrix24 group), then Test in the catalog will only be visible to members of that group.

Answer:

No, it does not take into account.

Answer:

Different roles can view test results in different sections:

• Moderator The application allows you to view test results in reports: Settings - Reports.
• Moderator with limited permissions, manager The employee can view it in the reports. The reports will be located in the left menu of the personal account.
• Employee who has passed the testIf the test option is selected where the employee can view results, they will be presented in the My Tests section when the test is expanded. Additionally, the employee can generate reports, but only for themselves.

Answer:

Update the application in the section Feed - Clear application cacheOr update your browser. If the tests do not appear, they were likely deleted by another moderator. The LMS system does not provide for and does not allow deletion of tests.

Answer:

Test results can be viewed in different locations, depending on the role of the person planning to view the test results: Testee, the manager of the test taker, the moderator.

Testee.

An employee who has taken the test can view the test results in two places:

• My tests - "Completed" or In Progress tab. Open the required Test, results will be displayed. The detail level of results will depend on the settings chosen during test creation.
• Reports. The Reports item is available in the left menu. You can only generate reports for yourself; other employees cannot be selected.

Employee's supervisor.

• Reports. The Reports section is available in the left menu. You can create any reports here. When selecting employees, a manager can choose any of their subordinates.

Moderator.

• Settings (gear icon top right) - Reports. A moderator can build any reports for any employees.

Answer:

If the Publish button is not active, check the following options:

• One of the required fields (name, time on Test, passing Score) is not filled;
• Specific symbols have been entered in the test name (e.g., a snowboarder emoji);
• Questions not added;
• No correct answers selected;
• No reviewers selected for open questions (if any);
• In the test catalog, there is Test with the same name.