Matchmaking System Design

Overview

With the transformation of the financial services market, the demand for core technology for conducting financial transactions has grown. The financial transaction model has changed from traditional manual bidding to matchmaking orders through an electronic transaction system. This article aims to give a detailed background on the changing financial transaction, and the matchmaking technology that is driving these changes.

Background: The Evolving Financial Transactions

The traditional financial transactions mainly occur in the tangible financial market. The contracting parties of financial transactions reach an agreement through bidding, price negotiation, and other methods. Meanwhile, the transaction is complete upon making a physical delivery at the designated point as stipulated in the trade order.

Since humans primarily execute the entire transaction process, the disadvantages of traditional financial transactions include low efficiency and speed, restrictions on transaction time venue, high transaction costs, and security threats like insider transactions, poor transaction expansibility, and man-made errors.

With the development of technology, electronic transactions are preferred for financial trades. They have not only eliminated the challenges associated with the traditional transaction methods but also led to the rapid development of the modern financial industry. Key advantages of electronic financial transactions include high transaction efficiency and speed, high system security, no limits on transaction time and venue, and multi-directional expansion.

Therefore, the electronic transaction has become the mainstream financial transaction method. With the constant increase of transaction population and quantity, the system is subject to growing pressure. In the case of system faults during the transaction, the resulting loss is often inestimable. As such, a more reliable and efficient electronic transaction system is urgently needed.

Matchmaking transactions play a very important role in the financial transaction system. To design a matchmaking system, it is crucial to understand the nature and business of matchmaking transactions.

Financial Transactions: Overall System Design

Hierarchical Design

The financial transaction matchmaking system includes the following core modules:
●User: the end user is entrusted with providing the quotation and quantity, with the next being order generation, and sending the order to the transaction platform.
●Gateway: collects and distributes the user's order to the matchmaking engine.
●Matchmaking engine: it is the core of the transaction system. It receives and fulfills the order by following the business logic, matches and generates the transaction record, and feeds the transaction results back to the user.
●Database: used to save order and transaction data in the transaction process, and realize data persistence.

Matchmaking modules exist in different business divisions depending on the type of financial transactions. Each division is subject to independent matchmaking and hence immune from other divisions.

Hierarchical Design 1: Matchmaking Transaction Algorithm

As shown in Figure 1, the core business module of a matchmaking engine is the matchmaking transaction algorithm. Its goal is to execute the matchmaking function for the customer's order, and to ensure fairness, high efficiency and expandability of the algorithm. Since the matchmaking business varies for different financial transaction systems, this section outlines the commonly used algorithms.


Figure 1

Hierarchical Design 2: Order Queue

An important part of transaction matchmaking is the sale or purchase of orders. Creation of transaction records occurs by matching the selling and buying orders. Therefore, if the orders are not matched immediately, the system should save the data in a buying queue and a selling queue. The queue shall adhere to the principle of "price first," and "time first" in the case of the same price. The buying queue arrangement follows the sequence from low to high according to the entrusted prices, and the selling queue arrangement follows the sequence from low to high according to the assigned prices as shown in the figure below.


Figure 2

Matchmaking Sequence

When receiving a new buying order, the matchmaking engine will search from the beginning of the selling queue to confirm if any selling orders comply with the price discipline. All orders with a selling price lower or equal to the buying price are extracted from the queue and matched into a transaction. If the selling queue is empty or the queue head fails to meet the price relationship, the system inserts the buying order into the buying queue. It will then arrange and sort the newly inserted order in the buying queue.

Correspondingly, when receiving a new selling order, the matchmaking engine will search the beginning of the buying queue to confirm if any buying orders comply with the price discipline. All orders with buying prices higher or equal to the selling price are extracted from the order queue and matched into a transaction. If the buying queue is empty or the queue head fails to meet the price relationship, the system inserts the selling order into the selling queue. Since the selling queue has two parameters, price and time, both do sorting and insert the new order accordingly [23].


Figure 3

In combination with the selling or buying orders, Figure 3 shows the process followed by the matchmaking algorithm. According to the sequence illustrated in Figure 3, the order of the buying and selling queues is the basis for ensuring certainty in the algorithm's output.

Hierarchical Design 3: Memory Matchmaking

The poor performance of the database matchmaking technology is due to excessive interactions with the database, which leads to high I/O transactions. Meanwhile, the database transaction logic seems to restrict the overall processing speed of the system.

Hence this paper discusses the memory-based matchmaking technology. After minimizing database interaction, the system implements the matchmaking logic in the memory (as shown in Figure 4). Therefore, there are less I/O interactions in comparison with database matchmaking, and the performance is improved significantly. Memory volatility is the main disadvantage of memory matchmaking. In the case of accidental shutdown of the server, all transaction data is lost, and the reliability and consistency of the system reduce dramatically. Consequently, multi-machine hot standby and distributed consistency technology improve the reliability of memory matchmaking technology. They realize the high performance of memory matchmaking and data persistence of database matchmaking technology.


Figure 4

Hierarchical Design 4: Multi-Machine Hot Standby

The multi-machine hot standby technology improves fault tolerance because memory matchmaking shows indigent reliability and consistency when the matchmaking engine is abnormal. Furthermore, the financial transaction system has extremely low tolerance for interrupted service and data loss due to business operations being affected.

With the help of multi-machine hot standby, the system deploys a group of matchmaking engines into a mutual standby matchmaking engine cluster. Additionally, there is only one engine in service at any given time. When one of the operating engines breaks down and ceases to work, the engine cluster detects the fault and selects a standby engine to take over the task of the faulty engine. The adoption of multi-machine hot standby technology stems from the need to avoid long-term matchmaking interruption to ensure reliable services of the system. As shown in Figure 5, several matchmaking engines conduct hot standby to make sure that when the host matchmaking engine breaks down, the switchover between the host and the standby units occur within an acceptable time, with the standby unit guaranteeing seamless service.


Figure 5

The risk of an unavailable system from a single engine fault has significantly reduced. However, it remains hard to provide hundred percent availability due to service unavailability during large-scale failures in the server cluster. In the actual production environment, three mutual backup servers can provide higher reliability.

Hierarchical Design 5: Memory State Machine Replication

Since several mutual hot standby matchmaking engines are involved, it is necessary to ensure data consistency among servers according to the matchmaking system design and logic requirements. Therefore, consistency among several servers is essential.

The matchmaking algorithm acts as a certainty state machine, tailored into multiple copies and deployed to numerous matchmaking engines in the system. Each copy operates from the same initial status. While the system receives the order sent by the gateway, it also matches such orders, successively producing the transaction record, and updating the independent state of the algorithm.

Through this method, when the system operates normally, each matchmaking engine copy has the same result. In the event of a fault or exception in the system, the engine will is inconsistent. In other words, the system deems any inconsistent results as an exception.

Key Technologies

Following are the major technology divisions of the matchmaking system:
1.The state machine service for the certainty matchmaking algorithm is often deployed into several independent engines.
2.Received gateway orders serve as the input of the state machine for the certainty matchmaking algorithm.
3.According to its demand, you should select the order sorting method.
4.Each engine will match the order already sorted.
5.The transaction record output by the state machine for certainty matchmaking algorithm will serve as the response to the user or the database.
6.Monitor the status or output difference of the engine copy.

Implementation Scheme

To implement the matchmaking system replicated based on the memory status, the following schemes help realize the key technologies of state machine replication:

●The atomic multicast will solve the reliable multicast and global ordering of the engine order.
●The use of assembly line matchmaking technology based on the unlocked order queue to realize rapid order matchmaking.
●The realization of database interactions by asynchronous consistent persistence technology [3]
●The use of invalid redundancy technology to monitor the status of the engine cluster to ensure the fault tolerance of the system; [24] [3115) progress catching-up technology is used to solve the problems of recovering the fault matchmaking engine or adding a new engine.

Hierarchical Design 6: System Architecture

System Hardware Architecture

Figure 6 illustrates a standard performance matchmaking model's hardware architecture. The system is composed of N server sets, N gateway sets, X product clusters (each cluster consists of between 2 to 3 matchmaking engines, and responsible for response processing of product orders), one transaction record database and optional monitoring system. The client has an established connection to the corresponding gateway. The gateway is responsible for receiving orders submitted by the client and forwarding them to the relevant product cluster according to the financial product category of the order. All engines in the project cluster will receive the order sent by the gateway, match the order according to the matchmaking business rules, and relay the feedback message to the gateway and the client. Meanwhile, the transaction record generated by matchmaking will become part of the transaction record database.


Figure 6

By enlarging the product cluster and increasing the quantity of the engine, you can strengthen the reliability of the product cluster. It will forward different financial products to different matchmaking product clusters to realize efficient parallel matchmaking.

System Software Architecture


Figure 7

As shown in Figure 7, the elements that make up the high reliability and performance matchmaking model include the presentation layer, the forwarding layer, the business layer and the data layer. Matchmaking engine clustering mainly has the business layer at its core. Each engine is subject to the matchmaking process after order sequencing is done by the atomic multicast. A subsequent combination with unlocked order queues then delivers efficient assembly line matchmaking. Finally, the local log displays the results. In the entire business process, the messaging bus plays the role of returning the message to the forwarding layer. The system forwards the order to the corresponding engine cluster according to the product forwarding rules. The engine then reads the transaction record in the local log, writes it into process communication along with the asynchronous persistence on the data layer, and finally makes it persistent in the database. The local log strengthens the reliability of the system data. After a fault occurs, it is possible to recover the data from the local log; asynchronous persistence mechanisms improve the data persistence throughput rate.

Matchmaking Engine Architecture


Figure 8

To improve system maintenance, the matchmaking engine has an atomic multicast order sequencing module, a matchmaking processor module, and a transaction record log module along with memory data. Each module has inbuilt divisions designed according to the functional business. Each part has the following functions:
●Transaction order receiving thread: receive orders from the gateway and finish the multicast order sequencing process.
●Sending thread of trade order: send the sequenced order to relevant matchmaking business thread.
●Sending thread of trade information: provide the feedback about the status of order transaction to the gateway.
●Logic thread of peripheral activities: conduct standby processes for matchmaking data, and update memory order data.
●Logic thread of the matchmaking business: match the received orders according to the certainty matchmaking algorithm.
●Release the threads of exchange quotation: process memory exchange quotation information and release it to the gateway.
●Writing thread of synchronization logs: synchronously make the transaction record produced by order matchmaking persistent into a local log file.
●Agent process of asynchronous persistence: asynchronously read out the data in the log document and make the data persistent into a database.
●Order information: save the relevant price, quantity, user, limitation, type, status and other information of the order.
●Exchange quotation information: exchange quotation information in the process of the transaction.

System Interface

The matchmaking system mainly provides ordering and inquiry service, real-time feedback function of the exchange quotation and monitoring and checking service for the system status. As such, the reserved interfaces that the system ought to realize mainly include the ordering interface, order inquiry interface, quotation inquiry interface, system control interface, and operating state inquiry interface.

Conclusion

Starting with the overall design, the system transfers the matchmaking transaction processing from the database into the memory. Meanwhile, the multi-machine hot standby technology solves the volatility problem of memory matchmaking technology. Finally, a different approach suggests that the memory state machine replication scheme can serve as the route to realizing a high-reliability and high-performance matchmaking system.

With the transformation of the financial services market, the demand for core technology for conducting financial transactions has grown. The financial transaction model has changed from traditional manual bidding to matchmaking orders through an electronic transaction system. This article aims to give a detailed background on the changing financial transaction, and the matchmaking technology that is driving these changes.

时间: 2024-12-10 02:14:51

Matchmaking System Design的相关文章

Matchmaking System Design – Answering Frequently Asked Questions

With the transformation of the financial services market, the demand for core technology for conducting financial transactions has grown. The financial transaction model has changed from traditional manual bidding to matchmaking orders through an ele

leetcode 新题型----SQL,shell,system design

leetcode 主要是一个针对北美的coder人群找工作的代码练习网站,我在2015年初次接触这个网站的时候,总共只有200多道题目,是一个类似acm 的a题网站.这些年变化越来越大,主要是因为找工作当然是多样化的考核过程,leetcode 也逐渐与时俱进,推出了下面几个类别的联系,今天我们随便挑几个练习一下: 175. Combine Two Tables -SQL Table: Person Column Name Type PersonId int FirstName varchar L

分布式系统(Distributed System)资料

原文地址:https://github.com/ty4z2008/Qix/blob/master/ds.md 希望转载的朋友,你可以不用联系我.但是**一定要保留原文链接**,因为这个项目还在继续也在不定期更新.希望看到文章的朋友能够学到更多. <Reconfigurable Distributed Storage for Dynamic Networks> 介绍:这是一篇介绍在动态网络里面实现分布式系统重构的paper.论文的作者(导师)是MIT读博的时候是做分布式系统的研究的,现在在NUS

GFS - The Google File System

The Google File System http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.125.789&rep=rep1&type=pdf http://www.dbthink.com/?p=501, 中文翻译   Google牛人云集的地方, 但在设计系统时, 却非常务实, 没有采用什么复杂和时髦的算法和机制  设计大型系统, 最重要的就是, 简单和可靠, 复杂就意味着失控... 在设计GFS, 首先一个选择就是,

wince C#调试出现:0x80070070: 磁盘空间不足

用C#做界面,效率比MFC要高的多. wince 下,如果需要制作界面,或者需要访问sqlce数据库,C#就非常方便了,如果用MFC,你会发现,访问sqlce 数据库需要设置很多COM接口很麻烦.   初次使用C#开发智能设备应用程序,也就是在wince上运行的应用程序,用VS2005把C#界面程序写好之后,然后编译调试下载. 出现了0x80070070: 磁盘空间不足   错误信息如下:   ------ 已启动生成: 项目: File_cp, 配置: Debug Any CPU ------

Asp.net2.0之自定义控件ImageButton

上个星期三开始学自定义控件,做了不少练习.花了一上午时间写了一个imageButton,以前就像写这个控件,只是不会. 图片 正文 这个控件模仿winform中的button,可以支持图片和文字.可以选择执行服务器端程序还是客户端程序,还有一些简单的设置. 不足的是不支持样式,下次希望可以写一个工具条. 以下就是代码 以下为引用的内容: using System; using System.Collections.Generic; using System.Linq; using System.

经验丰富的设计师总结:交互设计的几种主要方法

文章描述:天才设计最可能被经验丰富的设计师所采用,他们已经经历过各种类型的问题,并能够从以前的项目中总结出解决办法. 开始学习交互设计的时候,常常有一些名词缩写,样子差不多,但是又不完全一样,让人容易混淆.比如UE,UED,UX,UXD,UCD--看到这些个缩写的时候,不难理解第一个U应该指的是用户(User),D指的是(Design),但是中间那个字母是什么意思呢?其实,这些名词缩写都是指的交互设计的几种主要方法. 其中UE(D)=UX(D)=User Experience Design(用户

ASP.NET2.0中实现图像转换过滤效果

asp.net|转换 本文描述一种创建定制Web控件的容易方法,并把该控件使用于一个ASP.NET 2.0 web页面中显示微软DirectX图像转换过滤效果. 一.简介 本文描述一种创建定制web控件并应用于一个ASP.NET 2.0 web页面中显示微软DirectX图像转换过滤效果的容易的方法.文中包括一个拥有11个不同控件的类库,每一个控件分别展示微软DirectX图像转换过滤效果的某些方面.在这11个控件中,5个是页面过渡效果控件,其它的6个是用于增强文本外观的过滤效果控件. 在这6个

关于数据仓库的十个最长问的问题

数据|问题 Although there are various approaches to data mining that seem to offer distinct features and benefits, many may not be powerful enough to meet your corporate knowledge discovery needs. But in fact just a few fundamental questions can quickly c