How can AI Improve your Workplace Efficiency?
-Automating Employee Onboarding-
Recently, AI has been rapidly developing and many AI tools have been created, but are you using AI in your business? Many companies may still be concerned about the hurdles to using AI in their business operations from the viewpoint of security, such as information leakage. In such a situation, learningBOX Corporation will release a new function "AI Assist" for the e-learning system "learningBOX" that is linked to ChatGPT (scheduled for September 20, 2023). We hope you will take this opportunity to try this function as it allows you to utilize AI easily and safely.
For this issue, we asked learningBOX members to cooperate with us and actually experience AI assist. In this article, we present an interview about "Report Analysis.
Click here to read the interview about "automatic generation of teaching materials.
What is "AI Assist" in learningBOX?
First, let us briefly introduce the new "AI Assist" feature.
Report Analysis...AI analyzes submitted reports based on model answers. Feedback comments are automatically generated to assist in the report grading process.
Automatic generation of teaching materials (quiz/test generation, memorization card generation)...Quizzes and memorization cards can be automatically generated with a single click from videos and PDF materials uploaded to learningBOX, website URLs, etc.
This enables time-saving question creation and report grading, which tend to be time-consuming, and assists in the smooth operation of online in-house training and school education.
Back to Table of Contents
Using "Report Analysis." Interviews
We interviewed the following two members of the Support Section of the Sales Department.
From left, Takashima (from Tokyo Branch Office), Kashiwabara (from Head Office)
Interviews were conducted via an online connection between Tokyo and the head office. Assuming a customer support (CS) training situation for new employee training, we asked the participants to use "report analysis" for tasks from creating training questions to scoring.
CS (Customer Support) training is one of the new employee training programs offered by learningBOX. It is a report assignment in which participants learn how they should respond as learningBOX customer support staff, based on actual customer inquiries. The report is submitted using the learning management system of learningBOX, and a senior employee grades each question and gives feedback. This is a difficult training program.
What problems have scorers had with report assignments so far?
What problems have you two had in grading reports?
Takashima: When grading reports, I not only have to grade them as pass/fail, but also write feedback comments, but I had to create easy-to-understand sentences from scratch while adding hints so that the students could derive the answers themselves. This was difficult and time-consuming every time.
In addition, each company member received more than 50 answers in total, and since I had to make pass/fail decisions and provide feedback on that volume in between work, before I knew it, I had a stack of reports waiting to be graded.
Kashiwabara: The responses in the report were phrased differently from person to person, so it took time to check each one to make sure I was getting the points right.
It used to take 5 minutes to grade one question, but it was reduced to 1 minute?
How did you actually use the report analysis? What were the good points?
Kashiwabara: First of all, I thought, "The age of AI has finally arrived! I thought (laughs). Putting that aside, I was glad that the AI automatically presents the similarity of the answers to the model answers and creates feedback comments so that I can immediately judge whether I have correctly filled in the correct points.
This seemed to reduce the time it used to take me from five minutes to grade one question to a sensible one minute or so. For example, if there are 10 questions, I might be able to reduce the time from 50 minutes to 10 minutes.
The feedback generated by the AI is also accurate, with such comments as "this sentence is missing" and "it is well written" for each point, so I have the impression that it can be used without problems as a basis for judgment.
Also, if you check the "Analyze on receipt of report" box in advance, the AI-analyzed answers are already displayed when you open the scoring screen, which I thought was very convenient.
If you check the "Analyze on receipt of report" checkbox, the AI analysis is already completed when you open the grading screen, which saves you time!
Takashima: I also liked how easy it was to operate and how quickly I was able to create report assignments. Basically, all I had to do was add model answers to the normal flow of creating questions. I thought it was a time saver overall because it also showed the analysis quickly.
As Kashiwabara-san said, the feedback sentences provided by the AI were also very accurate and helpful. For example, even if an answer lacks explanation, it will include specific advice in the text. It was very smooth because I could just copy the text and edit it.
▼The preparation of report assignments is also so speedy!
▼ Results screen with report analysis
It could be used not only for in-house training, but also for various examinations and test preparation!
You tried using it in the report grading scene of CS training this time, but what other situations could you use report analysis in?
Takashima: In addition to universities, qualification schools, and in-house training, industries that require accuracy, such as the nursing care and construction industries, have report assignments after training, so it could be used in those situations as well. I also thought it could be used for other purposes, such as employment and promotion exams, as well as in test preparation.
Kashiwabara: LearningBOX also has a diagnostic test function, but we sometimes receive comments from customers that it is difficult to create a diagnostic test. Certainly, it is difficult to create a diagnostic test from scratch unless you have studied statistics, etc. After using AI Assist, I feel that it is very good at analysis, so I thought it would be even more convenient if AI Assist was also included in the diagnostic test.
What were your concerns and points for improvement?
Did you have any concerns or areas for improvement after actually using the product?
Kashiwabara: It may be improved in future modifications, but currently, the bottleneck may be that you can only enter up to 1,000 characters in response to a report assignment. For example, I felt it was not enough for a university report assignment.
I also feel that there needs to be a basis for the "similarity to model answers" figure. This is because, if you answer the model answers as they are, you will naturally get 100%, but if you try to create some answers that are missing some wording from the model answers, the result is that many of them are around 85%. There was some variation in the numbers, and I thought it would be more convincing if I could objectively see what the percentages were based on.
The percentage of correct answers is displayed as "similarity to the model answers."
Furthermore, we also tested what would happen if we tried to give an answer that was better than the model answer, and the correct answer rate was 85%. This is because the AI compares the answer to the model answer, but if a person had graded it, this would be a point to be added to the score. In this regard, I felt it was safe to take the stance that "the AI is only an assistant".
Also, there was a point where I thought I might be confused about the input format at first. The input format of how to submit the assignment, but please note that you cannot use the report analysis unless you select "text input".
To use report analysis, select "Text Input" and then check the "Use AI Report Analysis (Beta) (Text Input Only)" checkbox!
Takashima: In CS training, we often put URLs and images (screenshots) as hints for feedback on answers, so I would really like to see this evolve to be able to insert them!
Would you actually like to use this feature in your work in the future?
Kashiwabara: Yes, by all means! There are more than 50 questions in total in the LearningBOX and QuizGenerator report assignments, so it is quite hard to grade them while doing daily work.
At this point, I feel that it is difficult to completely rely on AI to grade reports. However, it still saves a lot of time, so we definitely want to use it in the future. I am planning to use it in earnest during the CS training after I join the company in the summer!
Back to Table of Contents
SUMMARY
In this issue, you have seen an interview with Takashima and Kashiwabara about report analysis.
The good news is that by making effective use of AI with the viewpoint that "AI is only an assistant," it will be possible to efficiently perform tasks that used to take a long time!
Although there are still some issues to be addressed regarding the accuracy of the AI, we are confident that it will be useful in assisting with the grading of report assignments.
We are now offering a free "Easy Quiz Generation by AI" service on the QuizGenerator page, which allows users to generate quizzes automatically by simply entering a short topic or a URL containing the contents of a quiz. Please enjoy the AI experience here as well until the release of the AI assist function in September.
We also recommend this service! Also read
Back to Table of Contents