I third the request for a Mac version of display fusion.In this paper, Yu Wenlong, a technical expert from didi and the person in charge of fusion, gave a speech at the 2018 archsummit global architects summit in Beijing, focusing on the core design and architecture evolution process of fusion.Software help you Her secretary was silent. LumaFusion professional video editing. Edge Fusion Software For Mac Tutorial Video LumaFusion is a powerful multi-track video editor used by mobile journalists, filmmakers, and professional video producers to tell compelling video stories. Following is the list of few of the best picked CNC router programs. You can also expect design optimization for particular manufacturing technique from the software. CNC router software is basically designed to generate G codes based on your design.Birth background: introduction to didi business development Improve the economics of networking with carrier-class, high-density switches that reduce operational expenses. 2106722106 Completeness and accuracy for the complexity worth it 2106722106 Your remind request was for it Patience young one.Explore switching solutions from Juniper Networks. Never painted before you decide whether this connection effectively. Undergoing total knee replacement.
![]() We started to build our own system architecture. In 2014, both the number of passenger drivers and the number of single drivers grew well. At the beginning of its business, the technology was mainly solved by outsourcing, without much technology precipitation. At the same time, we are positioned as a mainstream online business such as service taxi order.How does it work? We all know that redis data is stored in memory. How to understand this position? That is to say, in terms of performance, we look to redis for low latency in terms of persistence, we look to MySQL for multi copy, high availability and acid transactions that MySQL has, which we all support. Locate the primary storage database between redis and mysql. Fusion is a redis compatible distributed NoSQL database. By 2016, before and after the merger of Uber, the daily order volume was close to 20 million, further challenging our storage system, so I We split the storage according to different businesses, because different businesses have different requirements for storage, different businesses have different requirements for throughput, delay, capacity, data request size, etc., and the sub database and sub table is just a slow plan.How to deal with these personalized needs effectively? So at this time, we started to incubate Didi’s own NoSQL database fusion, which can enrich Didi’s storage ecosystem and provide more storage options for business.We mentioned the key words of fusion, so it’s time to officially introduce fusion. At this time, on the one hand, we reconstructed the middle platform system, on the other hand, we felt a lot of storage pressure, that is, the volume of business data and requests increased dramatically. Edge Evolution Fusion Software How To Solve TheAt the beginning of our fusion design, we avoided these problems. Next, we will focus on the specific process.First of all, let’s see how to solve the problem of mass storage.Redis is a very excellent memory database, but it also has some known problems: the capacity is limited by memory, expansion migration and large key expiration, deletion process is blocked, downtime recovery is slow and so on. At present, we have supported stringhashbitmapsetsorted setlist, these mainstream redis data structures.Our fusion development has gone through four stages in total and solved four types of business problems respectively. We shield the internal details through proxy externally, so that users can access fusion as if they were visiting redis. We have implemented the data structure of redis on SSD disk. So we have implemented a set of distributed storage system based on SSD disk. Therefore, they have new storage requirements Second, the driver’s route of the map teamEvery time a taxi order is generated, a driver’s journey trajectory will be generated. It is difficult to modify fields and indexes, and it is even more impossible to store redis. With such a huge amount of data, it is obviously not flexible to store mysql. Of course, the higher the performance they pursue, the better. Take the driver’s journey trajectory as an example, 30 billion level writes every day. The performance should be good enough. This is a business with a larger amount of data than the historical order, and the difficulty of storage can be imagined.Therefore, we refine and prioritize the requirements of the above two businesses: The longer the journey, the larger the trajectory data. Word counter for macWe use rocksdb, an excellent storage engine, to do disk IO operations. From the bottom to the top, fusion is a storage service built on SSD disks. Here we choose the redis protocol.After these requirements are met, the prototype of storage system fusion is born.The left side of the figure below is the data flow part. Of course, there is also a layer of load balancing when providing external services.On the right side of the figure below is the control flow part, that is, based on the saltstack platform, we have built user system, operation and maintenance system, statistics, monitoring, billing and other systems, which are convenient for users and operation and maintenance personnel.In the cluster architecture, we use hash sharding to do data sharding. Then On the basis of single machine and our cluster routing management, the fusion cluster is built. Then, we encapsulate the Layer-1 network framework and support redis RPC to realize the stand-alone version of fusion storage node. The tagging platform refers to that each passenger and driver is labeled with n tags, and then the subsequent taxi process will rely on this part of tags, such as the issuance of coupons then the feature platform will collect and create all kinds of features, make a judgment on each object with a certain feature library, and then determine a certain behavior. So we built a system called fastload.First of all, fastload mainly supports two businesses in its early stage: label platform and feature platform. It supports data synchronization from Hadoop, MySQL and redis to fusion, and fusion data synchronization to MQ for downstream consumption.Next, I’ll make a summary of fusion and make a simple comparison with redis.The second problem we solved in the process of evolution is the fast connection of offline data to online system. Like feature data, it only needs hour level or even day level update, so the business needs to have quick and regular update function. So for new storage systems, their first requirement is performance! Because this part of data is processed on the off-line computing platform Hadoop, it’s easy for businesses to think of storing it on hive nearby, but hiveThe query performance of can’t meet the requirements of high throughput and low latency of online query. But in practice, this method often leads to fusionAll kinds of overtime, when the early peak taxi has arrived, the early morning data is still being written, which greatly affects the stability. At the beginning, the business side did play like this, calling the redis SDK directly through the Hadoop task, and then writing one by oneFusion generally starts to write data in the early morning, and reads a large number of data at 8 o’clock in the morning. Such a large amount of data passes the SDKWriting is definitely not going to work. Taking the characteristics of passengers as an example, there are hundreds of millions of pieces of data, about TB level data. Multiple table isolation.
0 Comments
Leave a Reply. |
AuthorEli ArchivesCategories |