FennelMerokaDocs

NOTE: Project Meroka is dead in the water. This page is simply archived here

Project Meroka

An R&D project to explore the viability, suitability and performance of Fennel enhancements for query processing on commodity compute devices exposed using OpenCL standards. OpenCL presents a unique opportunity to fully exploit CPU and GPU resources to perform extraordinarily fast data processing using both Task and Data parallelism available on specialized devices (GPUs with hundreds of processors) and generalized devices (CPUs utilizing L2 cache effectively).

Meroka could, eventually and upon satisfying criteria listed below, find its way into the mainline LucidDB Fennel branch if the general x86 OpenCL libraries prove to be worthwhile, in and of themselves.

The project is named after an abbreviation of the German word, Mehrrohrkanone, meaning multi-barrelled gun. The SIMD capabilities of OpenCL allow data processing in batches, effectively triggering and firing in parallel.

Links

• OpenCL Wikipedia article
• OpenCL Overview Slides
• AMD StreamSDK w/ OpenCL for GPUs/CPUs
• OSX OpenCL
• MORE

Source and Infrastructure

To ensure that this R&D project does not interfere with normal build, QA, collaboration we've setup google code SVN repository and issue tracking.

SVN: hosted at code.google.com https://meroka.googlecode.com/svn/trunk/ (//open/dev@13357 branched)

ISSUE TRACKING: http://code.google.com/p/meroka/issues/list

WIKI: Eigenpedia FennelMerokaDocs

EMAIL LIST:

Sponsorship and Timeline

DynamoBI is the primary sponsor on the project, along with support from Eigenbase and SQLstream architects for design, architectural, and code reviews. Two research students from Alchemy research team at INRIA will be the primary developers/researchers on the project.

Each Milestone will include a review, comment, and analysis period by all stakeholders.

Feb / Mar 2010

Milestone 1: Inclusion of OpenCL and internal use in one XO

Milestone 2: Use of OpenCL in approx 3 XOs

Jun / Jul 2010:

Milestone 3: Experimentation and prototype of OpenCL aware scheduler enhancements (FennelMerokaScheduler)

Milestone 4: Enhancements to FennelMerokaScheduler to handle task/memory affinity for OpenCL algorithms

Milestone 5: Wrap up, documentation, testing on other devices

NOTE: Milestones 1 and 2 will run on OpenCL x86, and GPU chosen by researchers. Milestones 3-4 will run on GPU chosen by researchers. Milestone 5: will run on as many reasonable implementations on our device list.

Project Success Criteria

• Large improvements in individual XO performance (5-20X improvement)
• Aggregate SSB benchmark improvements, using FennelMerokaScheduler, of at least 5x on x86 and 20x on GPU.
• Ability to run, seamlessly, OpenCL x86 implementations on any machine (win32, lin32, lin64, OSX). Primary goal is to avoid having to maintain two implementations of an XO object.
• Ability to mix OpenCL enabled and standard XO objects in the scheduler, easily

People and Involvement

• Nicholas Goodman of DynamoBI - Project Lead and Sponsor
• Soufiane Baghdadi of INRIA, FR - Primary Researcher/Developer
• Baghdadi Riyadh of INRIA, FR - Primary Researcher/Developer
• John V. Sichi of Eigenbase - Technical Reviewer, LucidDB data/storage/executor/scheduler expert
• Julian Hyde of SQLstream - Technical Reviewer, Scheduler/Optimizer expert