Internet-based Test with learningBOX reduced the costs by 1 million yen!

Release Date :April 22, 2024

• IBT/CBT

Release Date :April 22, 2024

Tsuhan Expert Association conducts certification exams such as the "Tsuhan (Mail Order) Expert Certification'' as part of its direct marketing human resource development efforts. We also provide seminars, training, and consulting services. When administering the test, in addition to the traditional method of taking the test at a test center, we have been offering the internet based test(IBT) with learningBOX since 2021. This time, the head of the association reveals why they chose learningBOX to go online, how they are using it, and some of the challenges they are facing.

Pay-as-you-go pricing system is flexible and scalable.

First of all, could you tell us about your business?

We are developing direct marketing human resources by offering certification examinations and other programs for those who are engaged in businesses such as D2C (*1), which sell products directly to consumers without using intermediary distributors.

The certification examinations have been offered since 2009, and a cumulative total of more than 10,000 people have passed the examinations. Currently, six different examinations are offered twice a year, in spring and fall: Levels 3, 2, and 1 of the Mail Order Expert Certification Test, and the specialist courses "Customer Centricity," "Mail Order CX Management," and "Fulfillment CX.

I am working for the director in charge of the secretariat and also I am the one who chose learningBOX.

1 D2C (DtoC) stands for Direct to Consumer, a business model in which products planned and manufactured by the company itself are sold through its own e-commerce site or directly managed stores without intermediary distribution.

Can you tell us about the background of the introduction of learningBOX?

We started using learningBOX in 2021 with the dedicated server plan (currently called Enterprise). I wanted to not only simply offer e-learning, but also manage examinees.

When searching for an LMS (Learning Management System) with the key words of "free" and "LMS," learningBOX came up at the top of the list.

I liked the simple pay-as-you-go pricing model. Many LMSs require a large initial cost, a fixed monthly or annual fee, or, especially with on-premise systems (where the company builds and operates its own servers, communication lines, and systems), which the cost can run into the millions of yen easily.

With learningBOX, if the number of test takers gradually increases, we can increase the number of accounts by increasing the number of subscribers. I also wanted to use my own domain name, which is also available as a paid option.

There were several other candidates that came up, but when limited to a pay-as-you-go service as well as using own domain name, learningBOX was the only one left. We thought this was exactly the service we were looking for.

How are you actually using the learningBOX?

We have been using for the certification examinations. Our certification examinations had been conducted exclusively by CBT (Computer Based Testing) at test centers nationwide, operated by a company specializing in contracted testing. learningBOX was introduced to add the option of IBT (Internet Based Testing), which is a testing method conducted via the Internet, to these tests, which can be taken online.

Currently, we operate in two ways: at test centers and online. Approximately 1,000 to 1,500 people take the test each year in the spring and fall, and 30% to 40% of them take the test online at home using learningBOX.

Currently, students are required to purchase a paper textbook for the exam and study for it. learningBOX allows students to take a summative test that reflects the content of the paper textbook before the exam, and if they pass the test, they can take the exam. We set Prerequisites between those tests and exams.

The certification test then creates the exam questions as content and sets a passing score in the creation form option. If the student exceeds that standard score, he or she passes the exam. We are also in the process of posting all of the paper textbooks in PDF format starting with the next exam in the fall.

Satisfied with the support I can rely on when I need help.

What do you feel are the challenges in operating learningBOX?

Regarding the prevention of cheating, as I mentioned earlier, we require that students take a summative test before taking the certification test on the learningBOX. Only those who have taken and passed the summative test are allowed to take the certification exam, so we include an element of identification here. To be honest, there are 100 questions and you have 60 minutes to solve them, so if you need to cheat, you will not be able to solve them in time.

However, there is also a demand for having the test taken at a test center after proper verification of the applicant's identity. In some cases, the summative test itself is a requirement for promotion or an internal qualification system of a company, and in such cases, some HR manegers are concerned about taking the test online because it is related to personnel evaluations. 60% of our association's test takers are sent from their companies, and about half of them are considered that the results of the tests are part of their in-house employees evaluation system.

There are services that use a camera on the computer to check the movements of the user and determine if the user is cheating, but we have heard some people say that even a slight movement can be detected and not sufficient enough yet.

That's right. In fact, we would like to see learningBOX used for various certification exams, and we are currently developing a function for identity verification, etc., which we plan to announce soon. We hope you will consider this as well.

Great news. In a few more years, it will be commonplace to have AI authentication and cheating prevention in place when taking the exam at home.

There is one more issue: we are in the process of creating an integrated database to manage all past examinee data. If all the examinees took the exam online, we would be able to manage the data on the LMS, but in the case of our association, some examinees take the exam at test centers, so we are planning to create a database that will include information on more than 10,000 examinees from 2009 to the present.

We would like to manage the history of each test taker and also create a "My Page" for each test taker.

This is because we are receiving an increasing number of inquiries regarding the certifications they have earned. We have introduced a certification renewal system, which allows you to keep your certification number for another two years if you take the content for renewal every two years. So we have been getting a lot of questions about the certifications they have obtained in the past and which ones they should renew in the future. To address this, we feel it is necessary to create a database and create a "My Page" where you can check your certification history and other information yourself.

That's a lot of work. learningBOX will support digital badges (digital certificates that electronically prove your qualifications and learning history) in the future. Digital certificates and certifications issued in accordance with a common global technical standard are called "Open Bdges" and can be centrally managed in your own "Open Badge wallet. We hope you will consider this option as well.

So, are there any other issues that have been raised by your examinees?

Yes, there have been people who have been unable to take the certification test because they unintentionally terminated in the middle of the test, and we have made it so that they cannot take the test again after one termination, but if they were told it was a reception problem, they were allowed to retake the test.

We have 120-130 questions registered, from which 100 will be randomly selected. You will see similar questions, but there is no reason to be skeptical there. But when we were contacted about such cases, we called the support desk of learningBOX and asked them to follow the logs. Every time I logged in at what time and minute, I checked what happened at this time.

How was the response of the support desk at that time?

I have been very satisfied with the response every time. No matter who answers the phone, even if it is not the person in charge, the answer is always polite and courteous. Recently, I have seen many services that do not respond to phone calls, only emails. learningBOX has been very helpful in answering my calls as soon as I called with a problem. Everyone's response has been excellent.

Not only about logs, but also when I had problems in creating questions, I called them and they taught me a lot of things. I immediately asked how to set up Prerequisites (clear conditions) in the summative test before taking the exam, or even how to create a test if I had any trouble. The intuitive UI is very easy to use, and I think you can get used to the detailed settings very quickly if you use it. In the beginning, especially if I had a problem, I would call them immediately for help. Thank you very much for that.

IBT using learningBOX is reasonable than CBT

Have you noticed any benefits since the implementaion of learningBOX?

Sure. There was the COVID-19, but the number of test takers did not decrease. Even under the COVID-19, the test center was operating with restrictions, but some people switched to taking the test from home with the learningBOX.

In fact, since the introduction of learningBOX, about 30% to 40% of all examinees have used the IBT method from home. In terms of cost, I think we were able to reduce the cost by less than 1 million yen because of the way we combined the the basic contract and spot contract, if necessary.

This is a per person cost issue related to test administration, but it is actually higher for CBT implementation compared to learningBOX. For example, let's say you subscribe to learningBOX with a standard plan for 100 people and add 300 accounts with a spot contract. If you use the full plan with 400 people, the cost per person would be about 1,400 yen. learningBOX is about 1,000 yen cheaper per person compared to CBT.

I am assuming, though, that you are utilizing spot contracts and managing the number of people diligently. This has probably saved us a little less than 1 million yen.

Do you have any requests to improve our service?

I am using the standard plan, but the upload limit per material is 500MB. 500MB is not enough for high quality and long videos, and it would be helpful if it were possible to compress the video and upload it in a quality that is easy to view.

I am thinking of making it mandatory to watch the video content as well as the PDF material before the exam in the next fall or later. However, that video is 3 hours long and I am trying to figure out how best to upload it.

I see. We also recommend that you split up your videos and upload them. Finally, what advice would you give to those who want to start the certification test or are considering the IBT?

Well, I found the contract format very convenient and flexible to accommodate an increase or decrease in the number of people when I introduce the system. I think learningBOX is ideal for a small start. If we simply accumulated IDs, the number would explode, so by making good use of spot contracts and operating without unnecessarily increasing the number of IDs, we were able to reduce costs.

What we were also struggling with was that during the time when the test was administered only on paper and the data of test takers who took the test at the test center was stored in disparate files. As I mentioned earlier, we are currently working hard to create an integrated database. So, for those who are just starting out, for example, we recommend that you consult with us from the beginning about how you want to manage test takers before you begin.

In addition, learningBOX will be equipped with an identity verification system in the future, which will make it even easier to use.

Thank you for your participation
!