Performance tips: Configure NXPowerLite Server for speed

Option 1: Run key processing tasks on a faster disk (SSD)

NXPowerLite performs its optimization processing in the %TEMP% folder of the user that the NXPowerLite service is logged in as. Its central state database is stored and accessed in the %ProgramData% folder on the machine NXPowerLite is installed on. These both require intense disk I/O and act as the main speed bottleneck. To significantly increase speed you need to:

1. Install NXPowerLite on a machine that uses an SSD as its main drive ensuring the %ProgramData% folder is also on the SSD.
2. Configure NXPowerLite to use a custom Temp folder located on an SSD.

Effect: Roughly 2x speed increase

Option 2: Increase optimizer threads + Option 1

By default, NXPowerLite is configured to use 2 optimizers, which means 2 files from the scanned list, can be processed in parallel. The more optimizers that are used the more CPU & RAM resources are used.
To further increase the speed you can increase the number of optimizer threads from the default value of 2.

Increasing threads the number of configured threads to more than 4 will only increase speed if the Temp Folder and ProgramData are located on a faster disk, otherwise disk I/O will throttle any increase in threads.

Large datasets

For large datasets, we recommend using multiple installations of NXPowerLite all configured to process discrete subsets of the entire data. This means the processing will be performed in parallel, bringing the time for completion down considerably.

Remote deployment / Network bandwidth

If NXPowerLite is installed on a system where the data to reduce is not stored locally, please be aware that there will an increase in network traffic. Before processing a file, NXPowerLite copies each file to the machine on which it is installed. After processing, it will overwrite the original. This increases the network traffic and means the performance of the software may be limited by the available network bandwidth.

Speed benchmarks

In our testing with v8.0.12 on a typical dataset, NXPowerLite will compress around 1TB of new data in a 24hr period. Any data that has previously been seen by NXPowerLite will be skipped making subsequent runs through data only consider 'new' files making it much faster. The testing was performed using a machine with the following specifications: