東華大學圖書館 |

Parallel Fuzzy K-Means Clustering on Top of an Olap Database /

Record Type:	Electronic resources : Monograph/item
Title/Author:	Parallel Fuzzy K-Means Clustering on Top of an Olap Database // Αθανάσιος Κιούρτης, Athanasios Kiourtis.
Author:	Κιούρτης, Αθανάσιος,
other author:	Kiourtis, Athanasios,
Description:	1 electronic resource (121 pages)
Notes:	Source: Masters Abstracts International, Volume: 84-03.
Contained By:	Masters Abstracts International84-03.
Subject:	Software. -
Online resource:	https://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=29298114
ISBN:	9798845748683

Parallel Fuzzy K-Means Clustering on Top of an Olap Database /
Κιούρτης, Αθανάσιος,

Parallel Fuzzy K-Means Clustering on Top of an Olap Database /Αθανάσιος Κιούρτης, Athanasios Kiourtis. - 1 electronic resource (121 pages)

Source: Masters Abstracts International, Volume: 84-03.

The era we live in is attacked by the big data phenomenon. Multiple enterprises store large amounts of data for analysis, making the field of data analysis more and important. But how is someone able to gain insights from this massively evolving data? There comes the data mining field, which is what this thesis investigates, and especially the clustering field, whose algorithms can group unknown data into multiple clusters.Needing to perform clustering in a distributed way, the framework of Hadoop is analyzed, which offers a way to execute parallel programs on a cluster of machines, storing the data in a distributed file system. The parallel executions are achieved using the MapReduce paradigm which transforms complex computations over a set ofpairs, so that many jobs can run together. As for the distributed storing, it is achieved using the Hadoop distributed file system (HDFS), a file system that provides scalable and reliable data storage. One of the projects that uses Hadoop for performing clustering, classification and collaborative-filtering techniques to large data, is Mahout.Among all the clustering algorithms, the thesis gives details about a Fuzzy clustering algorithm, Fuzzy kMeans, which groups the data to clusters, by assigning to each data point different degrees of association for each cluster.The main problem that has to be solved is how we can cluster data stored in an OLAP database, in a parallel way. For that reason, it is studied how Mahout implements the fuzzy kMeans algorithm using the Hadoop's components, and after that, the thesis proposes a "similar approach" of the algorithm, running on top of an OLAP database.In more details, the new fuzzy kMeans implementation instead of MapReduce is using a three-steppedpair idea (Map, Reduce & Final: Reduce). In the first step, the clustering jobs are split and assigned to different threads which export their ownpairs. In the second step, each thread, without waiting for the other threads to finish their first step, continues using the extractedpairs from its previous step and exports its own. The last step, takes place when all of the threads finish the aforementioned two steps, and the results of the second step are merged together, to produce the final clustering result. In addition, instead of using the HDFS for storing purposes, the implementation is making use of an OLAP database.For the previous idea, a prototype implementation has been developed and preliminary tests were run, comparing the different fuzzy kMeans clustering implementations.Finally, the thesis concludes with the fact that more attention should be given to the data that evolve day by day, as mining and extracting information out of it, becomes more and more complicated, whereas it could be considered as a very challenging and attractive job.

English

ISBN: 9798845748683Subjects--Topical Terms:

619355
Software.

Parallel Fuzzy K-Means Clustering on Top of an Olap Database /
LDR:07613nmm a22004093i 4500 001 2437632
005 20251124054949.5
006 m o d
007 cr|nu||||||||
008 261011s2015 miu||||||m |||||||eng d
020 $a 9798845748683
035 $a (MiAaPQD)AAI29298114
035 $a (MiAaPQD)Piraeus7821
035 $a AAI29298114
040 $a MiAaPQD $b eng $c MiAaPQD $e rda
100 1 $a Κιούρτης, Αθανάσιος, $e author. $3 3828687
245 1 0 $a Parallel Fuzzy K-Means Clustering on Top of an Olap Database / $c Αθανάσιος Κιούρτης, Athanasios Kiourtis.
264 1 $a Ann Arbor : $b ProQuest Dissertations & Theses, $c 2015
300 $a 1 electronic resource (121 pages)
336 $a text $b txt $2 rdacontent
337 $a computer $b c $2 rdamedia
338 $a online resource $b cr $2 rdacarrier
500 $a Source: Masters Abstracts International, Volume: 84-03.
500 $a Advisors: Πρέντζα, Ανδριάνα; Peris, Ricardo Jimenez; Prentza, Andriana.
502 $b M.Sc. $c University of Piraeus (Greece) $d 2015.
520 $a The era we live in is attacked by the big data phenomenon. Multiple enterprises store large amounts of data for analysis, making the field of data analysis more and important. But how is someone able to gain insights from this massively evolving data? There comes the data mining field, which is what this thesis investigates, and especially the clustering field, whose algorithms can group unknown data into multiple clusters.Needing to perform clustering in a distributed way, the framework of Hadoop is analyzed, which offers a way to execute parallel programs on a cluster of machines, storing the data in a distributed file system. The parallel executions are achieved using the MapReduce paradigm which transforms complex computations over a set ofpairs, so that many jobs can run together. As for the distributed storing, it is achieved using the Hadoop distributed file system (HDFS), a file system that provides scalable and reliable data storage. One of the projects that uses Hadoop for performing clustering, classification and collaborative-filtering techniques to large data, is Mahout.Among all the clustering algorithms, the thesis gives details about a Fuzzy clustering algorithm, Fuzzy kMeans, which groups the data to clusters, by assigning to each data point different degrees of association for each cluster.The main problem that has to be solved is how we can cluster data stored in an OLAP database, in a parallel way. For that reason, it is studied how Mahout implements the fuzzy kMeans algorithm using the Hadoop's components, and after that, the thesis proposes a "similar approach" of the algorithm, running on top of an OLAP database.In more details, the new fuzzy kMeans implementation instead of MapReduce is using a three-steppedpair idea (Map, Reduce & Final: Reduce). In the first step, the clustering jobs are split and assigned to different threads which export their ownpairs. In the second step, each thread, without waiting for the other threads to finish their first step, continues using the extractedpairs from its previous step and exports its own. The last step, takes place when all of the threads finish the aforementioned two steps, and the results of the second step are merged together, to produce the final clustering result. In addition, instead of using the HDFS for storing purposes, the implementation is making use of an OLAP database.For the previous idea, a prototype implementation has been developed and preliminary tests were run, comparing the different fuzzy kMeans clustering implementations.Finally, the thesis concludes with the fact that more attention should be given to the data that evolve day by day, as mining and extracting information out of it, becomes more and more complicated, whereas it could be considered as a very challenging and attractive job.
520 $a Actualmente estamos viviendo un momento que esta sufriendo el fenomeno "big data". Muchas empresas almacenan enormes cantidades de datos para su posterior analisis, haciendo que el campo del data mining este adquiriendo mayor importancia. Pero, ¿como puede ser alguien capaz de obtener informacion de estos datos que evolucionan de forma masiva? Es aqui donde entra en juego el campo del data mining, sobre el cual se centra el estudio de esta tesis y en especial el campo del clustering, cuyos algoritmos permite agrupar datos que a simple vista no tienen relacion entre ellos.Debido a la necesidad de ejecutar estos algoritmos de clustering de manera distribuida, se va a utilizar Hadoop, un sistema distribuidos que ofrece la posibilidad de ejecutar programas en paralelo sobre un conjunto de maquinas. La ejecucion paralela se consigue mediante la utilizacion del paradigma MapReduce el cual transforma computaciones complejas sobre un conjunto de pares clave-valor de tal manera que diversas tareas puede ser ejecutadas conjuntamente. Para conseguir el almacenamiento distribuido de los datos, se esta utilizando HDFS, un sistema que provee escalabilidad y fiabilidad al almacenamiento de los datos. Uno de los proyectos que utiliza Hadoop para la ejecucion de tecnicas de clustering, clasificacion y filtrado colaborativo sobre grandes cantidades de datos es Mahout.Entre todos los algoritmos de clustering existentes, en esta tesis se ha utilizado un algoritmo fuzzy, en concreto Fuzzy k-means, el cual agrupa los datos en grupos mediante la asignacion a cada dato de un grado de asociacion con cada uno de los grupos.El principal problema a resolver es como se puede aplicar el algoritmo de clustering sobre una base de datos OLAP de manera paralela. Por esta razon, se ha estudiado como Mahout implementa el algoritmo Fuzzy K-Means utilizando los componentes que ofrece Hadoop y a continuacion proponer aproximaciones similares del algoritmo que permitan ser ejecutadas sobre una base de datos OLAP.Concretamente, la nueva implementacion del algoritmo Fuzzy K-Means utiliza, en vez de MapReduce, un concepto de pares clave-valor basado en tres pasos (Map, Reduce y FinalReduce). En el primero paso, el proceso de clustering es dividido y asignado a diferentes procesos los cuales exportan sus propios pares clave-valor. En el segundo paso, cada proceso, sin esperar al resto de procesos a que terminen el primer paso, continuan utilizando los pares clave-valor del paso anterior y exportan son propios pares clave-valor. El ultimo paso tiene lugar cuando todos los procesos han terminado los dos pasos anteriores y los pares clave-valor resultantes del segundo pasos de todos los procesos son integrados, dando lugar al resultado final del clustering. Ademas, en vez de utilizar HDFS como sistema de almacenamiento, se ha utilizado una base de datos OLAP.De esta manera, se han realizado diversas implementaciones del algoritmo de clustering Fuzzy K-Means, para realizar posteriormente distintos experimentos que han permitido la comparacion entre ellos.Finalmente, se ha concluido que es necesario prestar mayor atencion a los datos que evolucionan constantemente, asi como a la extraccion de informacion ya que este proceso cada vez es mas complicado, dando lugar a ser considerado como una tarea desafiante y atractiva.
546 $a English
590 $a School code: 4463
650 4 $a Software. $2 gtt. $3 619355
650 4 $a Computer science. $3 523869
690 $a 0800
690 $a 0984
700 1 $a Kiourtis, Athanasios, $e author. $3 3828688
710 2 $a University of Piraeus (Greece). $e degree granting institution. $3 3767578
720 1 $a Πρέντζα, Ανδριάνα $e degree supervisor.
720 1 $a Peris, Ricardo Jimenez $e degree supervisor.
720 1 $a Prentza, Andriana $e degree supervisor.
773 0 $t Masters Abstracts International $g 84-03.
790 $a 4463
791 $a M.Sc.
792 $a 2015
856 4 0 $u https://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=29298114