The time of fundamental research and testing of various algorithms in the field of Big Data, apparently, has come to a logical conclusion: existing solutions based on Fast Data (fast data) are quietly making their revolution in processing huge volumes of requests and information in minimal time intervals, bringing the business closer to the idea of responses and process regulation in real time.
Of course, if you look at how any service with the use of such technologies works, it becomes clear that everything is not as simple as you might think when you call, for example, a bank or a transport company.
Even if you determine offhand how many services are involved in the company’s macro service: for example, when a client calls, he first gets to the voice center, where the subscriber is authenticated in several stages, then, with the full participation of the voice robot, intelligent recognition of the question, the answer, and the dialogue in general occurs.
The most interesting thing is then, that is, the process of making a decision on the issues that are voiced by the client: receiving a service, clarifying, offering a similar option, and – as an extreme case-switching to the operator in the absence of a ready-made solution in the robot algorithm.
Of course, this is an extremely simplified description of the process, there are also queries to various proprietary and public databases, search for intersections, anomalies, and much more.
That is, the priority of Fast Data in accessing and processing the information flow is obvious, machine learning has already reached the level where the system itself can take information from open sources to make a decision, correctly interpret it and make an independent decision.
The undoubted advantage of this approach is the speed of processing, when the information is processed not in bulk storage, but comes at the time of the request and, by reducing the number of internal processing of the storage and, most importantly, in the speed of data acquisition within the corporate network: if we take for an ideal request to receive data within 1 second, then we need to accommodate: the speed of making a decision about the request, sending, receiving, getting into the right ranking for the best speed in the priority of the response to our request, then-when all parameters are agreed – sending a request to the data store (if you are lucky – to the already cached information), receiving the response volume, forming a packet, and finally sending it to us, again taking into account the allocated speed for requests of this level.
That is, you need to understand that, despite the speed, for example, 1 terabit in the network, the packet will not physically go at this speed – other parallel data, splitting into parts and assembling at the final node will interfere.
That is, Fast Data makes it possible to significantly reduce the load on the system with the number of requests exceeding tens of thousands per second.
If you imagine that the service receives only 1000 requests per second, and correlate with the number of filters (in the same bank, for example) – that is, identification, identification, distribution, and these are just the names of the processes – and each contains a complex algorithm that generates many logical forks, checks, and its own internal requests at each step, then it becomes clear that such a data shaft within one center in the concept of “all in one” now does not stand up to any competition.
Therefore, virtualization is gaining momentum – the speed of data access and the bandwidth of internal and external networks are still limiting processes that Fast Data has successfully overcome so far.