数据仓库技术及应用项目6 基于函数实现微博和门户数据统计.ppt

任务2门户日志数据预处理3)对数据进行转化将截取好的数据转换为"yyyy-MM-ddHH:mm:ss"形式SimpleDateFormatformator=newSimpleDateFormat("dd/MMMMM/yyyy:HH:mm:ssZ",Locale.ENGLISH);Datedate=formator.parse(s);SimpleDateFormatrformator=newSimpleDateFormat("yyyy-MM-ddHH:mm:ss");rformator.format(date);完整代码如下：packagecom.simple;importjava.text.ParseException;importjava.text.SimpleDateFormat;importjava.util.Date;importjava.util.Locale;importorg.apache.hadoop.hive.ql.exec.UDF;publicclassHiveUDFextendsUDF{publicStringevaluate(Stringline){SimpleDateFormatformator=newSimpleDateFormat("dd/MMMMM/yyyy:HH:mm:ssZ",Locale.ENGLISH);SimpleDateFormatrformator=newSimpleDateFormat("yyyy-MM-ddHH:mm:ss");Stringlines=line.toString(); intgetTimeFirst=lines.indexOf("[");intgetTimeLast=lines.indexOf("]");Stringtime=lines.substring(getTimeFirst+1,getTimeLast).trim();Datedt=null;Stringd1=null;try{ dt=formator.parse(time); d1=rformator.format(dt); }catch(ParseExceptione){ //TODOAuto-generatedcatchblock e.printStackTrace(); } returnd1;}}任务2门户日志数据预处理2.对日志中的请求字段数据进行分割，并以输出1)创建HiveUDTF类，并继承GenericUDTF类，initialize,process,close三个方法packagecom.simple;importorg.apache.hadoop.hive.ql.exec.UDFArgumentException;importorg.apache.hadoop.hive.ql.metadata.HiveException;importorg.apache.hadoop.hive.ql.udf.generic.GenericUDTF;importorg.apache.hadoop.hive.serde2.objectinspector.ObjectInspector;importorg.apache.hadoop.hive.serde2.objectinspector.StructObjectInspector;publicclassHiveUDTFextendsGenericUDTF{ @Override publicStructObjectInspectorinitialize(ObjectInspector[]arg0)throwsUDFArgumentException{ //TODOAuto-generatedmethodstub returnsuper.initialize(arg0); } @Override publicvoidclose()throwsHiveException{ //TODOAuto-generatedmethodstub } @Override publicvoidprocess(Object[]arg0)throwsHiveExcep