Data Mining with Big Data

 Data Mining with Big Data

ABSTRACT:

Big Data concern large-volume, complex, growing data sets with multiple, autonomous sources. With the fast development of networking, data storage, and the data collection capacity, Big Data are now rapidly expanding in all science and engineering domains, including physical, biological and biomedical sciences. This paper presents a HACE theorem that characterizes the features of the Big Data revolution, and proposes a Big Data processing model, from the data mining perspective. This data-driven model involves demand-driven aggregation of information sources, mining and analysis, user interest modeling, and security and privacy considerations. We analyze the challenging issues in the data-driven model and also in the Big Data revolution.

EXISTING SYSTEM:

• The rise of Big Data applications where data collection has grown
  tremendously and is beyond the ability of commonly used software tools to capture, manage, and process within a “tolerable elapsed time.” The most fundamental challenge for Big Data applications is to explore the large volumes of data and extract useful information or knowledge for future actions. In many situations, the knowledge extraction process has to be very efficient and close to real time because storing all observed data is nearly infeasible.
• The unprecedented data volumes require an effective data analysis and prediction platform to achieve fast response and real-time classification for such Big Data.

 DISADVANTAGES OF EXISTING SYSTEM:

• The challenges at Tier I focus on data accessing and arithmetic computing procedures. Because Big Data are often stored at different locations and data volumes may continuously grow, an effective computing platform will have to take distributed large-scale data storage into consideration for computing.
• The challenges at Tier II center around semantics and domain knowledge for different Big Data applications. Such information can provide additional benefits to the mining process, as well as add technical barriers to the Big Data access (Tier I) and mining algorithms (Tier III).
• At Tier III, the data mining challenges concentrate on algorithm designs in tackling the difficulties raised by the Big Data volumes, distributed data distributions, and by complex and dynamic data characteristics.
  

PROPOSED SYSTEM:

• We propose a HACE theorem to model Big Data characteristics. The characteristics of HACH make it an extreme challenge for discovering useful knowledge from the Big Data.
• The HACE theorem suggests that the key characteristics of the Big Data are 1) huge with heterogeneous and diverse data sources, 2) autonomous with distributed and decentralized control, and 3) complex and evolving in data and knowledge associations.
• To support Big Data mining, high-performance computing platforms are required, which impose systematic designs to unleash the full power of the Big Data.

ADVANTAGES OF PROPOSED SYSTEM:

vProvide most relevant and most accurate social sensing feedback to better understand our society at real-time.

SYSTEM ARCHITECTURE:

 

SYSTEM CONFIGURATION:

HARDWARE CONFIGURATION:

Processor    -        Pentium IV

Speed                   -        1.1 Ghz

RAM          -        512 MB (min)

Hard Disk   -        20GB

Keyboard    -        Standard Keyboard

Mouse         -        Two or Three Button Mouse

Monitor      -        LCD/LED Monitor

SOFTWARE CONFIGURATION:

Operating System          -        Windows XP/7

Programming Language-        ASP .NET

Software Version           -        VISUAL STUDIO 2008

Database                        -        MSSQL

REFERENCE:

Xindong Wu, Fellow, IEEE, Xingquan Zhu, Senior Member, IEEE, Gong-Qing Wu, and Wei Ding, Senior Member, IEEE, “Data Mining with Big Data”, IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. 26, NO. 1, JANUARY 2014.

Panda: Public Auditing for Shared Data with Efficient User Revocation in the Cloud

Panda: Public Auditing for Shared Data with Efficient User Revocation in the Cloud

CLICK HERE TO VIEW THE OUTPUT

ABSTRACT:

With data storage and sharing services in the cloud, users can easily modify and share data as a group. To ensure shared data integrity can be verified publicly, users in the group need to compute signatures on all the blocks in shared data. Different blocks in shared data are generally signed by different users due to data modifications performed by different users. For security reasons, once a user is revoked from the group, the blocks which were previously signed by this revoked user must be re-signed by an existing user. The straightforward method, which allows an existing user to download the corresponding part of shared data and re-sign it during user revocation, is inefficient due to the large size of shared data in the cloud. In this paper, we propose a novel public auditing mechanism for the integrity of shared data with efficient user revocation in mind. By utilizing the idea of proxy re-signatures, we allow the cloud to re-sign blocks on behalf of existing users during user revocation, so that existing users do not need to download and re-sign blocks by themselves. In addition, a public verifier is always able to audit the integrity of shared data without retrieving the entire data from the cloud, even if some part of shared data has been re-signed by the cloud. Moreover, our mechanism is able to support batch auditing by verifying multiple auditing tasks simultaneously. Experimental results show that our mechanism can significantly improve the efficiency of user revocation.

EXISTING SYSTEM:

To protect the integrity of data in the cloud, a number of mechanisms have been proposed. In these mechanisms, a signature is attached to each block in data, and the integrity of data relies on the correctness of all the signatures. One of the most significant and common features of these mechanisms is to allow a public verifier to efficiently check data integrity in the cloud without downloading the entire data, referred to as public auditing (or denoted as Provable Data Possession ).This public verifier could be a client who would like to utilize cloud data for particular purposes (e.g., search, computation, data mining, etc.) or a third party auditor (TPA) who is able to provide verification services on data integrity to users. Most of the previous works  focus on auditing the integrity of personal data.

DISADVANTAGES OF EXISTING SYSTEM:

v Especially when the number of re-signed blocks is quite large.

v Existing users may access their data sharing services provided by the cloud with resource limited devices, such as mobile phones.

v Frequent Security Issues.

PROPOSED SYSTEM:

We propose Panda, a novel public auditing mechanism for the integrity of shared data with efficient user revocation in the cloud. In our mechanism, by utilizing the idea of proxy re-signatures, once a user in the group is revoked, the cloud is able to resign the blocks, which were signed by the revoked user, with a re-signing key. As a result, the efficiency of user revocation can be significantly improved, and computation and communication resources of existing users can be easily saved. Meanwhile, the cloud, which is not in the same trusted domain with each user, is only able to convert a signature of the revoked user into a signature of an existing user on the same block, but it cannot sign arbitrary blocks on behalf of either the revoked user or an existing user. By designing a new proxy re-signature scheme with nice properties, which traditional proxy re signatures do no have, our mechanism is always able to check the integrity of shared data without retrieving the entire data from the cloud.

ADVANTAGES OF PROPOSED SYSTEM:

v Easily Revocable of signatures for the existing users.

v The public verifier can audit the integrity of shared data without retrieving the entire data from the cloud.

SYSTEM ARCHITECTURE

SYSTEM CONFIGURATION:-

HARDWARE REQUIREMENTS:-

Processor                  -        Pentium –IV

Speed                        -        1.1 Ghz

RAM                         -        512 MB(min)

Hard Disk                 -        40 GB

Key Board                -        Standard Windows Keyboard

Mouse                       -        Two or Three Button Mouse

Monitor                     -        LCD/LED

SOFTWARE REQUIREMENTS:

Operating system      :         Windows XP.

Coding Language      :         .Net

Data Base                 :         SQL Server 2005

Tool                          :         VISUAL STUDIO 2008.

REFERENCE:

Boyang Wang, Baochun Li, and Hui Li, “Panda: Public Auditing for Shared Data with Efficient User Revocation in the Cloud” IEEE TRANSACTIONS ON SERVICE COMPUTING VOL. PP, NO. 99, December 2013

A Stochastic Model To Investigate Data Center Performance And Qos In Iaas Cloud Computing Systems

A STOCHASTIC MODEL TO INVESTIGATE DATA CENTER PERFORMANCE AND QOS IN IAAS CLOUD COMPUTING SYSTEMS

ABSTRACT:

Cloud data center management is a key problem due to the numerous and heterogeneous strategies that can be applied, ranging from the VM placement to the federation with other clouds. Performance evaluation of Cloud Computing infrastructures is required to predict and quantify the cost-benefit of a strategy portfolio and the corresponding Quality of Service (QoS) experienced by users. Such analyses are not feasible by simulation or on-the-field experimentation, due to the great number of parameters that have to be investigated. In this paper, we present an analytical model, based on Stochastic Reward Nets (SRNs), that is both scalable to model systems composed of thousands of resources and flexible to represent different policies and cloud-specific strategies. Several performance metrics are defined and evaluated to analyze the behavior of a Cloud data center: utilization, availability, waiting time, and responsiveness. A resiliency analysis is also provided to take into account load bursts. Finally, a general approach is presented that, starting from the concept of system capacity, can help system managers to opportunely set the data center parameters under different working conditions.

EXISTING SYSTEM:

In order to integrate business requirements and application level needs, in terms of Quality of Service (QoS), cloud service provisioning is regulated by Service Level Agreements (SLAs): contracts between clients and providers that express the price for a service, the QoS levels required during the service provisioning, and the penalties associated with the SLA violations. In such a context, performance evaluation plays a key role allowing system managers to evaluate the effects of different resource management strategies on the data center functioning and to predict the corresponding costs/benefits.

Cloud systems differ from traditional distributed systems. First of all, they are characterized by a very large number of resources that can span different administrative domains. Moreover, the high level of resource abstraction allows to implement particular resource management techniques such as VM multiplexing or VM live migration that, even if transparent to final users, have to be considered in the design of performance models in order to accurately understand the system behavior. Finally, different clouds, belonging to the same or to different organizations, can dynamically join each other to achieve a common goal, usually represented by the optimization of resources utilization. This mechanism, referred to as cloud federation, allows to provide and release resources on demand thus providing elastic capabilities to the whole infrastructure.

DISADVANTAGES OF EXISTING SYSTEM:

·       On-the-field experiments are mainly focused on the offered QoS, they are based on a black box approach that makes difficult to correlate obtained data to the internal resource management strategies implemented by the system provider.

·       Simulation does not allow to conduct comprehensive analyses of the system performance due to the great number of parameters that have to be investigated.

PROPOSED SYSTEM:

In this paper, we present a stochastic model, based on Stochastic Reward Nets (SRNs), that exhibits the above mentioned features allowing to capture the key concepts of an IaaS cloud system. The proposed model is scalable enough to represent systems composed of thousands of resources and it makes possible to represent both physical and virtual resources exploiting cloud specific concepts such as the infrastructure elasticity. With respect to the existing literature, the innovative aspect of the present work is that a generic and comprehensive view of a cloud system is presented. Low level details, such as VM multiplexing, are easily integrated with cloud based actions such as federation, allowing to investigate different mixed strategies. An exhaustive set of performance metrics are defined regarding both the system provider (e.g., utilization) and the final users (e.g., responsiveness).

ADVANTAGES OF PROPOSED SYSTEM:

To provide a fair comparison among different resource management strategies, also taking into account the system  elasticity, a performance evaluation approach is described. Such

an approach, based on the concept of system capacity, presents a holistic view of a cloud system and it allows system managers to study the better solution with respect to an established goal and to opportunely set the system parameters.

SYSTEM ARCHITECTURE:

SYSTEM CONFIGURATION:-

HARDWARE CONFIGURATION:-

     ü Processor               -       Pentium –IV

ü Speed                      -       1.1 Ghz

ü RAM                       -       256 MB(min)

ü Hard Disk                -       20 GB

ü Key Board               -       Standard Windows Keyboard

ü Mouse                      -       Two or Three Button Mouse

ü Monitor                    -       SVGA

SOFTWARE CONFIGURATION:-

ü Operating System                   : Windows XP

ü Programming Language         : JAVA

ü Java Version                   : JDK 1.6 & above.

REFERENCE:

Dario Bruneo, Member, IEEE-“ A Stochastic Model to Investigate Data Center Performance and QoS in IaaS Cloud Computing Systems”- IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS 2013.