手机版 欢迎访问it开发者社区(www.mfbz.cn)网站

当前位置: > 开发

Hudi 系列(二)- 编译 hudi

时间:2021/6/1 10:57:36|来源:|点击: 次

编译 hudi

如果是第一次编译 hudi 会花很长时间,maven 本地仓库为空的情况下,我花了40分钟(视网络环境)。编译步骤如下:

  • 下载 releace 源代码,https://hudi.apache.org/releases,现在的最新版本是0.7.0 。
  • 安装 maven ,maven 的版本选择 3.6.3 。
  • 配置 maven 的 setting.xml 为中央仓库增加里面增加 aliyun 的 mirror 仓库 。
  • 修改 hudi 项目 pom.xml 里面增加 aliyun repository 。
  • 删除项目 pom.xml 中和 repo.spring.io 相关的 repository ,不然编译会出错,详见:https://github.com/apache/hudi/pull/2481,https://github.com/apache/hudi/pull/2481/commits/f67acf4d51010704188977b5090ef9c20325860f 。
  • 执行 mvn package -DskipTests 开始编译 。

编译成功后,会看到如下信息:

[INFO] Reactor Summary for Hudi 0.7.0:
[INFO] 
[INFO] Hudi ............................................... SUCCESS [  5.237 s]
[INFO] hudi-common ........................................ SUCCESS [ 22.062 s]
[INFO] hudi-timeline-service .............................. SUCCESS [  2.297 s]
[INFO] hudi-client ........................................ SUCCESS [  0.136 s]
[INFO] hudi-client-common ................................. SUCCESS [  4.842 s]
[INFO] hudi-hadoop-mr ..................................... SUCCESS [  4.118 s]
[INFO] hudi-spark-client .................................. SUCCESS [  9.274 s]
[INFO] hudi-sync-common ................................... SUCCESS [  1.117 s]
[INFO] hudi-hive-sync ..................................... SUCCESS [  3.633 s]
[INFO] hudi-spark-datasource .............................. SUCCESS [  0.147 s]
[INFO] hudi-spark-common .................................. SUCCESS [  5.556 s]
[INFO] hudi-spark2_2.11 ................................... SUCCESS [  3.694 s]
[INFO] hudi-spark3_2.12 ................................... SUCCESS [  5.367 s]
[INFO] hudi-spark_2.11 .................................... SUCCESS [ 15.311 s]
[INFO] hudi-utilities_2.11 ................................ SUCCESS [  6.966 s]
[INFO] hudi-utilities-bundle_2.11 ......................... SUCCESS [ 21.478 s]
[INFO] hudi-cli ........................................... SUCCESS [ 12.717 s]
[INFO] hudi-java-client ................................... SUCCESS [  2.497 s]
[INFO] hudi-flink-client .................................. SUCCESS [  7.377 s]
[INFO] hudi-dla-sync ...................................... SUCCESS [  3.680 s]
[INFO] hudi-sync .......................................... SUCCESS [  0.069 s]
[INFO] hudi-hadoop-mr-bundle .............................. SUCCESS [ 20.377 s]
[INFO] hudi-hive-sync-bundle .............................. SUCCESS [  4.636 s]
[INFO] hudi-spark-bundle_2.11 ............................. SUCCESS [ 18.165 s]
[INFO] hudi-presto-bundle ................................. SUCCESS [ 11.876 s]
[INFO] hudi-timeline-server-bundle ........................ SUCCESS [ 11.303 s]
[INFO] hudi-hadoop-docker ................................. SUCCESS [  4.424 s]
[INFO] hudi-hadoop-base-docker ............................ SUCCESS [  1.532 s]
[INFO] hudi-hadoop-namenode-docker ........................ SUCCESS [  1.259 s]
[INFO] hudi-hadoop-datanode-docker ........................ SUCCESS [  1.476 s]
[INFO] hudi-hadoop-history-docker ......................... SUCCESS [  1.917 s]
[INFO] hudi-hadoop-hive-docker ............................ SUCCESS [  7.409 s]
[INFO] hudi-hadoop-sparkbase-docker ....................... SUCCESS [  1.876 s]
[INFO] hudi-hadoop-sparkmaster-docker ..................... SUCCESS [  1.319 s]
[INFO] hudi-hadoop-sparkworker-docker ..................... SUCCESS [  1.461 s]
[INFO] hudi-hadoop-sparkadhoc-docker ...................... SUCCESS [  1.776 s]
[INFO] hudi-hadoop-presto-docker .......................... SUCCESS [  1.577 s]
[INFO] hudi-integ-test .................................... SUCCESS [ 35.448 s]
[INFO] hudi-integ-test-bundle ............................. SUCCESS [03:10 min]
[INFO] hudi-examples ...................................... SUCCESS [ 28.104 s]
[INFO] hudi-flink_2.11 .................................... SUCCESS [ 13.175 s]
[INFO] hudi-flink-bundle_2.11 ............................. SUCCESS [ 24.294 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  08:42 min
[INFO] Finished at: 2021-01-27T16:35:12+08:00
[INFO] ------------------------------------------------------------------------

Copyright © 2002-2019 某某自媒体运营 版权所有