DeepGreen LLVM JIT SQL数据库技术新进展.pdf

DeepGreen LLVM JIT SQL数据库技术新进展 SQL LLVM JIT技术发展 SQL JIT,CodeGen历程 2003 MontDB 2008 MontDB/X100 2010 HyPer 2012 Cloudera Impala 2015 VitesseDB, DeepGreen 科研界SQL JIT,CodeGen 性能优势 科研界创新点 • just-in-time query compilation • query execution using SIMD instructions • multi-core aware parallelization • query execution on compressed data • rethinking I/O for modern storage (e.g. Flash) • adaptive tuple layouts during query processing • recycling of intermediates in non-materializing database systems and • column-oriented multi-dimensional clustering. LLVM介绍 SQL Exec ExecPlan LLVM JIT Parser Plan Interpreter Code Cache Table DeepGreen 简介 DeepGreen 基本架构 大规模并行处理 SQL MPP (Massively Parallel Processing) SQL 无共享架构Shared-Nothing Architecture MapReduce Master 节点 ... ... 生成查询计划并派发 汇总执行结果 Network Interconnect Segment 节点 ... ... 执行查询计划及数 据存储管理 外部数据源 并行装载或导出 DeepGreen 增强了20倍速度 • test=# select count(*) from tt; • count • • test=# set vitesse.enable=1; • 1000000 • SET • (1 row)