History of UNIX Project Build Tools

版权声明:本文为博主原创文章,未经博主允许不得转载。

History of UNIX Project Build Tools  

(The following is derived from the HACKING.txt file of the old open source project which I stopped supporting many years ago. R.I.P.)

You might have noticed above that there are SIX STEPS required to do a rebuild after editing configure.in. Why is it so complicated?

最初最初的时候,只有Makefile,并没有configure等工具出现。

You might remember the days when all the dependencies and rules were encapsulated in one file (called Makefile) and no matter what you changed (including the Makefile itself) the make command would figure out how to rebuild everything. That's not true anymore.

Unix版本越来越多,开始要求Makefile具备跨平台的功能:不同平台,对Makefile有不同的要求。这时工具链就必须延伸。

The original Makefile model worked well before the proliferation of many different types of UNIX and the advent of cross-platform compatibility.

To handle all the different types of Unix, the Makefile has to be complex — so complex that it is no longer practical to edit by hand. Furthermore, many things have to be figured out by the machine where the program is going to be built, rather than the machine where the programmer developed it. So, the "meta-rules" for creating Makefiles got too complicated to edit by hand. Eventually there got to be a lot of different types of source files, and a lot of different rules for what to do if you want to change something. The best way to explain this is by going through the history in chronological order — starting prior to the origin of Makefile and make.

为了理解工具链是如何一步步延伸的,就从历史先后顺序看起。

人肉CC搞定

Originally, all changes were made just by changing C source code (in the foo.c and foo.h files) and typing cc to compile it into a binary:

sources -> cc -> binary

用Makefile推导文件依赖,自动编译

Then came compiling and linking as a separate step. This created the situation that if you change just one foo.c file, you only have to re-compile that one file and then link, but if you change a foo.h file you probably had to compile everything. To save time, people figured out how to make a list of rules telling which ".c" files depend on which ".h" files. The make tool was developed, a program that would automatically figure out what needed to be recompiled. make takes a new source file, called Makefile, and also uses all the program source files. Rebuilding still consisted of just one step:

Makefile and sources

-> make -> binary

make wasn't quite as smart as it should have been. For example, there's no way to get it to check if Makefile itself was changed. To work around this, programmers started adding a "target" called "clean" (or something similar) that removes all the object files. Then, if you change Makefile (for example, after realizing you left out an "#include" dependency) you can type make clean to force it to remove the objects, and make to recompile everything. Commands like make clean are still used today. (花絮,这里解释了为什么改变了Makefile后应该执行make clean)

多种Unix变种出现,如BSD,AT&T System III,源码和Makefile中需要条件编译/配置(configuration),以适应不同系统。早起做法是把配置集中到一个头文件里面,例如config.h中,或放到Makefile里。

After a few years, different versions of Unix started to exist (like BSD vs. AT&T System III), and people noticed that you had to change the foo.c and foo.h files and Makefile in different ways depending on what type of Unix you were compiling for. Those changes were called "configuration". Manual configuration is tedious — it takes a lot of knowledge and diligence to make all the correct changes for your own particular type of Unix. Eventually it was decided all such changes should be controlled by #ifdef tests (like "|#ifdef BSD_4_1"), and the #defines could be specified in the Makefile| or a header file called config.h (or something similar). Building then required two steps. In the first step, you edit the Makefile to #define each of the defines (like BSD_4_1) that you need on your system:

[plain] view plaincopy

  1.           generic  
  2.         Makefile and  -.  
  3.           config.h      \  
  4. STEP 1.             manually-edit  
  5.                           \  
  6.                            `->  custom Makefile  
  7.                                  and config.h  
  8.   
  9.             Makefile  
  10.                  and   -.  
  11.              sources     \  
  12. STEP 2.                 make  
  13.                           \  
  14.                            `->  binary  

然后大家受不了这么山寨的搞法,发明了专门的配置系统(1992年),它首先探测系统类型,然后生成针对这个系统的一坨宏。这个系统的名字叫configure (Makefile的上一步出现!),它是个shell脚本。

Next, standard "configuration systems" were created. Usually a configuration system was a set of shell scripts that made all the tests and modifications automatically. For example, it is easy to test for BSD version 4.1, and everyone agreed to use BSD_4_1 to indicate you're on that system. So, (in theory) all it takes is one big set of instructions, (typically, a shell script called configure), to test for all the different types of hardware, oeprating systems, libraries, etc. and generate the #defines for that system. The result, for most programmers, was to replace the first manual step with something more automatic.

Because Makefile was now auto-generated by configure, the configure file was what you edited when you wanted to change the Makefile, and the Makefile became an uneditable, automatically-generated file just like the program binary. The two build steps became:

[plain] view plaincopy

  1.  configuration-files  -.  
  2.                         \  
  3. STEP 1.              configure  
  4.                           \  
  5.                            `->  Makefile  
  6.   
  7.             Makefile  
  8.                  and   -.  
  9.              sources     \  
  10. STEP 2.                 make  
  11.                           \  
  12.                            `->  binary  

起初configure要负责两件事:探测操作系统类型;生成Makefile。后来(1994年),这两件事分化为由两个工具来完成:autoconf负责探测操作系统类型;configure负责生成Makefile。autoconf的参数文件叫configure.in;configure的参数文件叫Makefile.in。

Several different types of configuration systems were in place by 1992. Some consisted of a script called configure that did all the tests to see what type of Unix you're running on, then generated the Makefiles. The configure script had to know a lot about the syntax of makefiles, as well as knowing a lot about how to test for different features of operating systems.

Eventually, the job of doing the operating-system tests and the job of creating the Makefiles from "Makefile templates" was split up into two different tools.

By 1994 it was generally agreed that the best tool for the operating-system tests was autoconf. It took one new source file: configure.in and generated a script called configure as output. This configure script, in turn, took one new source file called Makefile.in, and generated Makefile as an output file. At this point the build had three steps that worked like this:

[plain] view plaincopy

  1.           configure.in -.  
  2.                          \  
  3. STEP 0.               autoconf  
  4.                           \  
  5.                            `->  configure  
  6.   
  7. - - - - tarfile is distributed in this form - - - -  
  8.   
  9.           Makefile.in -.  
  10.                         \  
  11. STEP 1.              configure  
  12.                           \  
  13.                            `->  Makefile  
  14.   
  15.             Makefile  
  16.                  and   -.  
  17.              sources     \  
  18. STEP 2.                 make  
  19.                           \  
  20.                            `->  binary  

第一步用autoconf生成configure文件这件事一般只需要做一次,因为系统不会总是变,也不会经常增加对操作系统有新依赖的代码。回想一下,我们下载的很多开源包里面,只需要执行./configure; make; make install就行了,并没有autoconf什么事,就是这个原因。

Note that Step 0 only had to be done if you changed the configuration requirements, like if you added a major new feature that depended on something that is different on different systems (an example would be adding a graphical user interface to a program that was previously text-only). Therefore, the build process was now split into the "user installation" steps (steps 1 and 2) and the "complete rebuild from scratch" (steps 0 1 and 2). Typically, the programmer would perform step 0 and distribute the result to the users, who perform steps 1 and 2. This is indicated above where it says "tarfile is distributed in this form".

到autoconf这一温饱时代后,人们开始奔小康,对方便性提出了更高的要求:configure的输入文件Makefile.in太他妈大了啊!工程下面的所有文件都要写到这里面去,维护起来很费劲(不信你去找个Makefile.in来看看)。于是,大约在1996年,又发明了一个新工具:automake,专门用于生成Makefile.in。automake的输入文件叫Makefile.am。

The weak point in this system was Makefile.in. This had to be a very large and complex file, because it contained all the rules for how to generate a Makefile, and Makefiles were by this point very complex (about as complex as a programming language) and vary a lot from one OS to another. Since Makefile.in was a source file it had to be edited manually. Most of Makefile.in was the same regardless of what program you were building, and programmers found it cumbersome.

The solution to that was automake. It automatically creates Makefile.in from another new source file, called Makefile.am. By 1996, the standard build process had four steps (two for users doing an install and two more for people adding new features) and the steps were:

对于代码维护者,需要涉及到下图中4步;对于开源代码使用者,只需要涉及到后面两步。

[plain] view plaincopy

  1.           configure.in -.  
  2.                          \  
  3. STEP 0-A.             autoconf  
  4.                           \  
  5.                            `->  configure  
  6.   
  7.            Makefile.am -.  
  8.                          \  
  9. STEP 0-B.             automake  
  10.                           \  
  11.                            `->  Makefile.in  
  12.   
  13. - - - - tarfile is distributed in this form - - - -  
  14.   
  15.           Makefile.in -.  
  16.                         \  
  17. STEP 1.              configure  
  18.                           \  
  19.                            `->  Makefile  
  20.   
  21.             Makefile  
  22.                  and   -.  
  23.              sources     \  
  24. STEP 2.                 make  
  25.                           \  
  26.                            `->  binary  

插播一个问题:configure.ac在哪里?答:还没出生呢!

还记得configure.in是怎么来的吗?当年为了给configure减负,把操作系统探测的逻辑独立出来,搞了个autoconf工具。autoconf工具的输入文件叫configure.in。又过了几年,configure.in成了瓶颈,太他妈难写了,不光操作系统,什么库啊,驱动啊,都得管。为了解决这个问题,autoconf扩展了自己的功能,引入了一个叫做aclocal.m4的”宏文件“,这个文件是用一种叫”m4”的语言写的,擅长探测操作系统的方方面面。扩展后的autoconf,组合configure.in和aclocal.m4为输入,生成configure文件。

Over the next couple years, configure.in got bigger and included lots of code to test for lots of different types of libraries, drivers, operating systems, etc. Eventually configure.in became the biggest and hardest-to-maintain file, just like Makefile.in had been. More recent versions of autoconf have solved this by allowing for the use of a "macros" file called aclocal.m4. The "macros" are written in a language called m4, and they contain the rules for performing all sorts of different operating-system tests. As far as the build process is concerned, these can be treated as part of step 0-A, except that you don't ever have to worry about changing the contents of aclocal.m4:

[plain] view plaincopy

  1.           configure.in   
  2.             aclocal.m4   -.  
  3.                            \  
  4. STEP 0-A.               autoconf  
  5.                             \  
  6.                              `->  configure  
  7.   
  8. STEP 0-B.  (automake step, same as above)  
  9.   
  10. - - - - tarfile is distributed in this form - - - -  
  11.   
  12. STEP 1.    (configure step, same as above)  
  13.   
  14. STEP 2.    (make step, same as above)  

aclocal.m4可用aclocal工具直接生成,也可以手写。如果用工具生成的话,就需要引入macros文件作为aclocal工具的输入。

Around the same time it also became common to use a tool called aclocal to generate aclocal.m4, from a directory of macros files called "macros". This added a fifth step to the full build process:

[plain] view plaincopy

  1.           configure.in   
  2.            macros/*.m4   -.  
  3.                            \  
  4. STEP 0-A.            aclocal -I macros  
  5.                              \  
  6.                               `->  aclocal.m4  
  7.   
  8.           configure.in   
  9.             aclocal.m4   -.  
  10.                            \  
  11. STEP 0-B.               autoconf  
  12.                             \  
  13.                              `->  configure  
  14.   
  15. STEP 0-C.  (automake step, same as above)  
  16.   
  17. - - - - tarfile is distributed in this form - - - -  
  18.   
  19. STEP 1.    (configure step, same as above)  
  20.   
  21. STEP 2.    (make step, same as above)  

当新千年的钟声想起,我们终于告一段落,大厦落成。

This was the way things were done by around the year 2000.

Complete list of files and the order in which they are built:

[plain] view plaincopy

  1. ORIGINAL FILES  
  2.    
  3.         the file: configure.in  
  4.  is created from: typed in by hand  
  5.    
  6.         the file: Makefile.am  
  7.  is created from: typed in by hand  
  8.    
  9.         the file: src/adam.c  
  10.  is created from: typed in by hand  
  11.    
  12.         the file: src/adam.h  
  13.  is created from: typed in by hand  
  14.    
  15.         the file: src/anything.c      (any ".c" not listed below)  
  16.  is created from: typed in by hand  
  17.    
  18.         the file: src/anything.h      (any ".h" not listed below)  
  19.  is created from: typed in by hand  
  20.    
  21.    
  22.    
  23.  AUTO_GENERATED FILES  
  24.    
  25.         the file: config.h.in   
  26.  is created from: acconfig.h configure.in acconfig.h  
  27.               by: autoheader  
  28.    
  29.         the file: config.h  
  30.  is created from: config.h.in  
  31.               by: ./configure  
  32.    
  33.         the file: Makefile  
  34.  is created from: Makefile.in  
  35.               by: ./configure  
  36.    
  37.         the file: configure  
  38.  is created from: configure.in aclocal.m4  
  39.               by: autoconf  
  40.    
  41.         the file: aclocal.m4  
  42.  is created from: configure.in macros/*.m4  
  43.               by: aclocal -I macros  
  44.    
  45.         the file: Makefile.in  
  46.  is created from: Makefile.am  
  47.               by: automake  
  48.    

不过,还是那个问题:configure.ac在哪呢?configure.in还得手写?我嘞个去!

表着急,configure.ac只不过是configure.in的新名字而已啦:

At this time aclocal and AM_INIT_AUTOMAKE did not exist, so many things had to be done by hand. For instance, here is what a configure.in (this is the former name of the configure.ac we use today) must contain in order to use Automake 0.20:

原文地址:http://mrob.com/pub/comp/unix-building-history.html

补充阅读:

autoconf的历史:https://www.gnu.org/software/autoconf/manual/autoconf-2.65/html_node/History.html

automake的历史:ttps://www.gnu.org/software/automake/history/automake-history.html#Timeline

整个流程可视化:https://en.wikipedia.org/wiki/Configure_script

特别推荐看看automake的历史,里面有数位作者的回忆,从中可以了解到很多原委。

时间: 2024-10-02 07:40:10

History of UNIX Project Build Tools的相关文章

第 2 章 Build Tools

2.1. Apache Ant http://ant.apache.org/ 2.1.1. 安装 ant 2.1.1.1. 1.8 cd /usr/local/src wget http://mirror.bjtu.edu.cn/apache//ant/binaries/apache-ant-1.8.1-bin.tar.gz tar zxvf apache-ant-1.8.1-bin.tar.gz mv apache-ant-1.8.1 /usr/local/ cd .. ln -s apach

android studio-Android Studio 安装Installing Build Tools失败

问题描述 Android Studio 安装Installing Build Tools失败 安装更新失败,是不是因为被墙了的缘故?该怎么办呢? 解决方案 http://ask.android-studio.org/?/article/9 解决方案二: 自备梯子,重新下载安装. 解决方案三: 这个问题有两个解决方法,一是去修改build.gradle中的buildToolsVersion,改一个你本地有的版本:二是翻墙下载. 解决方案四: 安装Android Studio--Installing

bettermeans - an open and democratic project management tool

What is bettermeans? Bettermeans is an open and democratic project management tool. Unlike other project management tools that are based on tracking work in a command and control environment, BetterMeans allows you to work with your team in a more op

【转】Eclipse maven工程 Missing artifact com.sun:tools:jar:1.6.0:system 解决方法

解决方案一:通过maven取运行时参数,eclipse提供的环境变量,基本类似System.getProperty("java.home") <dependency>           <groupId>com.sun</groupId>           <artifactId>tools</artifactId>           <version>1.5.0</version>      

虚拟机-Android Studion (AS)相关版本对Project影响测试结论,亟盼大牛相助

问题描述 Android Studion (AS)相关版本对Project影响测试结论,亟盼大牛相助 这个问题我在一周内测试和各种网站查找资料,至少累计16小时,非常苦闷! 这个Project简单,来源于陈老师的一本书. App启动后,点击{登陆} (pic 1), 应该显示 Pic3, 但我这里显示 Pic2 . 我开始是 一行行自己手动输入的, 恐怕有错, 找陈老师给了源代码Copy各个 .xml, .java, 但是结果都相同. 我 在 AS里 New>import 陈老师的Project

Cocos2d-x Visual Studio Android Project

In the world of game development , Cocos2d is one of the biggest names out there. The open source Cocos2d framework has been the backbone of a vast number of top selling apps and games, provides support for a number of programming languages, and targ

Reapp 混合应用 - 帮助构建前所未所的强大应用(Reapp Hybrid apps - help you build powerful apps like never before)

Reapp 混合应用 - 帮助构建前所未有的强大应用 (Reapp Hybrid apps -  help you build powerful apps like never before) 太阳火神的美丽人生 (http://blog.csdn.net/opengl_es) 本文遵循"署名-非商业用途-保持一致"创作公用协议 转载请保留此句:太阳火神的美丽人生 -  本博客专注于 敏捷开发及移动和物联设备研究:iOS.Android.Html5.Arduino.pcDuino,否则

DelphiXE环境认知(第一章 Project Options)

DelphiXE环境认知 作者:帅宏军 时间:2011年4月 说明:根据DelphiXE的自带帮助翻译而来,为作者个人看法,如有翻译不当,仅供参考. shuaihj@163.com http://blog.csdn.net/shuaihj 第一章. Project Options Ø Project > Options 这个页面用来设置当前项目的Delphi编译器选项.Default勾选后可将当前的配置作为新工程的默认配置. 说明: 并不是这里所有的选项适用于所有类型的项目.例如,DCP out

解密方程式组织的Unix后门NOPEN

前言 不久之前,黑客组织ShadowBrokers(影子经纪人)曾声称他们从EquationGroup(方程式组织)那里窃取来了大量的黑客工具,并且他们还将部分工具放在网上进行拍卖. 近日,Vectra公司的安全研究专家NickBeauchesne对其中的一份泄漏文件进行了分析,并且发现了一个名叫"NOPEN"的Unix远程管理工具(RAT). Nick Beauchesne说到: "当我在对方程式组织泄漏的文件以及黑客工具进行分析时,我发现了一些非常有意思的东西,而这些东西