RayTurbo data improvement increases processing speed by 5 times

All range has improved the RayTurbo data, a monopoly data processing platform, and promised up to five times faster than the open source opponent Ray Data. These improvements aim to revolutionize large -scale data processing by reducing processing time and risk of operation.

Inspection of work levels to improve reliability

One of the most prominent features is to introduce checkpointing at work level designed to strengthen reliability in the production environment. This feature can resume reasoning workloads at the exact point of interruption due to manual or automatic cluster termination. RayTurbo data keeps the strict delivery schedule and competitive edge by keeping the cost of computing resources by preserving the execution status.

Unlike the existing Ray data that finances individual tasks when the operator node breaks down, the checkpointing of the RayTurbo can handle serious interruptions such as head node collisions or memory errors without the need for full restart. This development is particularly helpful for long -term batch reasoning, especially in the long -term arrangement that handles millions of records faced during the time or several days.

Vectorized cohesion for improved data analysis

RayTurbo data now supports completely vectorized aggregation, moving the calculation to the optimized default code in Python. This transition removes performance bottlenecks related to Python’s interpreters to improve the throughput of the latest CPU architecture. The new aggregation function is especially important for functional engineering and data summaries, especially when processing large data sets.

Optimized pipeline rules for efficient processing

In addition to the speed improvement, the Optimizer rules of RayTurbo Data have been upgraded to automatically reinforce the operation within the data pipeline, which focuses on filter and projection work. This optimization reduces unnecessary data processing so that the pipeline can be completed more quickly without changing the code you have written.

Performance benchmark and influence

Comprehensive benchmarks emphasize the performance benefits of rayTurbo data rather than open source layer data. In the test using the TPC-H Orders data set, RayTurbo has improved 1.6 times to 2.6 times for the aggregate workloads for pretreatment work related to filter and heat selection, and 3.3x to 4.9 times improved.

The test environment consists of one M7I.4XLARGE head node and a cluster with five M7I.16xlarge worker nodes, and object storage memory is set to 128GB per worker node. Such benchmarks emphasize the ability of RayTurbo Data, which can more efficiently handle large AI workloads, to provide a significant competitive advantage.

Image Source: Shutter Stock