Server construction

This is Nishimura from the development team. We have been using an instance of AWS (Amazon Web Services) for the development of learningBOX, but we are about to switch to development on our in-house server because of some inconveniences. Click here for the table of contents. 1. inconvenient truth about aws 2. considered commercial PCs... 3. assembled it 4. installing ProxmoxVE 5. steps to speed up the drive on Proxmox 6. moving an instance from an existing proxmox environment 7. so, did it get faster? 8. future plan Inconvenient truth about aws Convenient aws for operation, but not for development environment... Actually, it's not very fast. We mainly used T3 instances for development, but to be honest, T3 instances are not very fast, are they? If you use C5 series or Z1d, you can accelerate up to 4GHz, but honestly speaking, it is expensive. If you want to use multiple cores, the cost will be several times higher. Large instances are expensive. In fact, it may not be very fast, but using an instance with a large memory and fast CPU will cost you several tens of thousands of yen per month. There are no instances that are just right. If the memory required for development is 2.1 GB, you need an instance with 4 GB of memory. Isn't this wasteful? You may be able to get around this by using virtual memory or something, but the moment it no longer fits in the real memory, the processing capacity will drop by an order of magnitude. I want to use an instance of just the right size. I considered a commercial PC, but... I thought about buying a commercial PC and virtualizing it, but I just didn't have the right one. I don't need Windows. I don't need it. I can't remove it, or even if I do, it's not nearly as cheap. I don't need a GPU. Most PCs with fast CPUs have a high-performance GPU. You don't need one. (It may be necessary if desktop virtualization is installed, but this has not been tested.) I don't need a mouse or keyboard. I don't need it. I put it together. If you don't have just the right one, build it! So I assembled a PC for the first time in a year. The parts I used are as follows. All parts are shown except the case. For some reason, the case of a ping-pong racket is shown, but it has nothing to do with the case. Description of main parts Motherboard No particular preference CPU Core i9 10900 I chose this CPU for its single-threaded performance. I considered 10900K, but I thought 0.1GHz was not worth the 20,000 yen difference in price, so I decided to go with 10900. DRAM I installed 32GB of DRAM which is reasonably fast. I will add more if it is not enough. SSD I installed 1TB of a reasonably fast SSD (3GB/s). Power supply I chose a good power supply, as I was told by God that I should buy a good one. GPU Not included for now. Install ProxmoxVE Using an entire PC may be the fastest way, but that is not an effective use of equipment, so we built a virtual environment with ProxmoxVE. On the virtual environment, we are running Amazon Linux 2. The procedure for running Amazon Linux 2 will be introduced in another article. I thought the installation would be a piece of cake, but I wasted a lot of time because of a defective block on the USB flash drive I created for the installation. It would be better to make a habit of checking for bad blocks when making installation media. Steps to speed up the drive on Proxmox This is the one that good kids should not imitate, but you can improve disk IO on a virtual environment by enabling disk caching as much as possible. It will probably be faster than reading/writing directly without the virtual environment. However, good kids should not imitate this procedure, because if the power goes down before the write is completed, it will break. Moving an instance from an existing proxmox environment I got into a lot of trouble, but I won't go into it here because it would take too long to explain. If you read the error messages carefully, you will be able to manage most of the problems. If you don't understand what is wrong even after reading the error messages, you can usually figure it out by reading the source code. Do your best. Migration is not possible if the VM is linked to other VMs. You can't migrate if you are using a physical drive or mounting a device for CloudInit. Remove the device. If you haven't already created a Cluster in your company, create one in your existing Proxmox environment and add a new virtual environment to it. Frankly, I'm not very good at infrastructure, so we're looking for engineers who can manage this area for us! (Working at the head office in Tatsuno City, Hyogo Prefecture) So, did it get faster? We compared the time taken to test our learningBOX (automated backend testing with PHPUnit). Comparison results AWS environment (t2.small): 2.63 minutes Old Proxmox environment (Core i5 9600K): 1.22 minutes New Proxmox environment (Core i9 10900): 1.1 minutes The difference between the two environments below is not big, but the difference between the aws environment and the Proxmox environment is obvious. c5 and others improve a little more, but the difference between the Proxmox environment and the aws environment is still big. Operating frequency during auto-test When there is no load, it is stable at 2.8GHz, but when a load is applied, it instantly rises to around 5GHz. Faster auto-testing directly affects work efficiency. Whether or not strict TDD is performed, automated testing is indispensable for development above a certain scale. If a test results in an error, or if a new test is created, you cannot proceed to the next task until the automated testing process is complete. On a typical day, we may run a test 20 or 30 times per day, and if we can save a minute per test, we can save that much time to develop new features or improve code quality. Future Policies With the construction of the Proxmox environment, we now have two hardware units to support the virtual environment, and we have also achieved migration between virtual environments. However, three servers are required to build a HA (High Availability) environment, and we intend to build a third environment to realize a stable private cloud, even when the Zen3 generation of Ryzen is released.