Monday 19 October 2015

A GENERAL SCALABLE AND ELASTIC CONTENT-BASED PUBLISH/SUBSCRIBE SERVICE



ABSTRACT:
The big data era is characterized by the emergence of live content with increasing complexities of data dimensionality and data sizes, which poses a new challenge to emergency applications: how to timely disseminate large-scale live content to users who are interested in. The publish/subscribe (pub/sub) model is widely used to disseminate data because of its possibility of expanding the system to Internet-scale size. However, existing pub/sub systems are inadequate to meet the requirement of disseminating live content in the big data era, since their multi-hop routing techniques and coarse-grained partitioning techniques lead to a low matching throughput, and their upload capacities do not scale well. In this paper, we propose a general scalable and elastic pub/sub service based on the cloud computing environment, called GSEC. For generality, we propose a two-layer pub/sub framework to support the dissemination with diverse data sizes and data dimensionality. For scalability, a hybrid space partitioning technique is proposed to achieve high matching throughput, which divides subscriptions into multiple clusters in a hierarchical manner. Moreover, a helper-based content distribution technique is proposed to achieve high upload bandwidth, where servers act as both providers and coordinators to fully explore the upload capacity of the system. For elasticity, we propose a performance-aware provisioning technique to adjust the scale of servers to adapt to the churn workloads. To evaluate the performance of GSEC, about 1,000 servers are deployed and hundreds of thousands of live content items are tested in our Cloud Stack-based testbed. Extensive experiments confirm that GSEC can linearly increase the capacities of event matching and content distribution with the growth of servers, adaptively adjust these capacities in tens of seconds according to the churn workloads, and significantly outperforms the state-of-the-art approaches under various parameter settings
AIM
 The aim of this paper is to propose a two-layer pub/sub framework to support the dissemination with diverse data sizes and data dimensionality.
SCOPE
The Scope of this paper is GSEC can linearly increase the capacities of event matching and content distribution with the growth of servers, adaptively adjust these capacities in tens of seconds according to the churn workloads, and significantly outperforms the state-of-the-art approaches under various parameter settings.
EXISTING SYSTEM
Existing content-based publish/subscribe systems (pub/subs) are inadequate to satisfy the challenges of generality, scalability and elasticity. This mainly stems from the following three reasons. Firstly, most of the pub/subs are designed for disseminating either small-sized events or bulk content rather than give consideration to both sides. In contrast, the data sizes of current emergency applications may churn from tens of kilobytes to hundreds of megabytes, which makes these pub/subs not generic. Secondly, existing space partitioning and event routing techniques in content-based pub/subs do not scale well with the numerous skewed subscriptions and events. Existing space partitioning techniques lead to either unbalanced workloads among servers due to the coarse-grained partitioning  or high memory cost due to the over fine-grained partitioning. Besides, the multi-hop routing techniques lead to high disseminating latency. Thirdly, most pub/subs have no financial incentives to provide elastic event matching capacities or upload capacities according to churn workloads

DISADVANTAGES:

  1.  The existing pub/sub systems are inadequate to meet the requirement of disseminating live content in the big data era
  2.  Since their multi-hop routing techniques and coarse-grained partitioning techniques lead to a low matching throughput, and their upload capacities do not scale well. 

PROPOSED SYSTEM
In this paper, we propose a general scalable and elastic pub/sub service based on the cloud computing environment, called GSEC. For generality, we propose a two-layer pub/sub framework to support the dissemination with diverse data sizes and data dimensionality. For scalability, a hybrid space partitioning technique is proposed to achieve high matching throughput, which divides subscriptions into multiple clusters in a hierarchical manner. Moreover, a helper-based content distribution technique is proposed to achieve high upload bandwidth, where servers act as both providers and coordinators to fully explore the upload capacity of the system. For elasticity, we propose a performance-aware provisioning technique to adjust the scale of servers to adapt to the churn workloads. To evaluate the performance of GSEC, about 1,000 servers are deployed and hundreds of thousands of live content items are tested in our Cloud Stack-based test bed. Extensive experiments confirm that GSEC can linearly increase the capacities of event matching and content distribution with the growth of servers, adaptively adjust these capacities in tens of seconds according to the churn workloads, and significantly outperforms the state-of-the-art approaches under various parameter settings. 
ADVANTAGES

  1. GSEC can efficiently support the dissemination of diverse data sizes and data dimensionality in a general manner
  2. GSEC achieves more scalable performance with the increasing system sizes, and the performance-aware provisioning technique ensures continuous data dissemination service with a high performance price ratio.

SYSTEM ARCHITECTURE:




SYSTEM CONFIGURATION

HARDWARE REQUIREMENTS:-

·                 Processor               -   Pentium –III

·                Speed                -    1.1 Ghz
·                RAM                 -    256 MB(min)
·                Hard Disk         -   20 GB
·                Floppy Drive    -    1.44 MB
·                Key Board                 -    Standard Windows Keyboard
·                Mouse               -    Two or Three Button Mouse
·                Monitor             -    SVGA

SOFTWARE REQUIREMENTS:-

·                Operating System              : Windows  7                                       
·                Front End                  : JSP AND SERVLET
·                Database                  : MYSQL
·                Tool                           :NETBEANS



REFERENCE:
Ma, X, Wang, Y. “A General Scalable and Elastic Content-based Publish/Subscribe Service”, IEEE Transactions on Parallel and Distributed Systems, Volume 26,  Issue 8  AUGUST 2014.



No comments:

Post a Comment