secc
Distributed compiler with modern web technology
Last updated 3 years ago by ivere27 .
MIT · Repository · Bugs · Original npm · Tarball · package.json
$ cnpm install secc 
SYNC missed versions from official npm registry.

secc - Distributed compiler with modern web technology.

a project of 'Second Compiler'.

Key Features

  • RESTful API in Daemon/Scheduler
  • Preprocessed and Pump Mode
  • Memory Cache(works like remote Ccache)
  • Debug fission(-gsplit-dwarf option) supports.
  • Monitoring by Browser(Navigate to http://SCHEDULER:PORT)
  • Cross-compile(Various version/machine of compiler in network) supports
  • Gcc / Clang support

Quick Start

How to use

install by git clone (then npm install)

git clone https://github.com/ivere27/secc.git
cd secc
npm install

make sure you've already installed node.js, npm and (optionally) redis

  • Ubuntu $ sudo apt-get install nodejs npm nodejs-legacy redis-server
  • Mac $ brew install nodejs redis

Client - your PC

set PATH and NUMBER_OF_PROCESSORS(-jX)

edit "client" part in 'settings.json' file. set scheduler's address and port.

$ export PATH=/path/to/secc/bin:$PATH
$ export NUMBER_OF_PROCESSORS="8"
$ # then,
$ clang -c test.c

or just use CC, CXX and NUMBER_OF_PROCESSORS env
```sh
$ SECC_ADDRESS=172.17.42.1 SECC_CACHE=1 SECC_CROSS=0 \
CC=/path/to/secc/bin/clang CXX=/path/to/secc/bin/clang++ ./configure
$ SECC_ADDRESS=172.17.42.1 SECC_CACHE=1 SECC_CROSS=0 \
CC=/path/to/secc/bin/clang CXX=/path/to/secc/bin/clang++ make -j8

upload your compiler archive by ./tool/secc-upload-archive.js

$ # for clang
$ node secc-upload-archive.js --clang /path/to/clang /path/to/clang++ archivetool.js http://SCHEDULER:PORT
$ # for gcc
$ node secc-upload-archive.js --gcc /path/to/gcc /path/to/g++ archivetool.js http://SCHEDULER:PORT

in linux case(you can use the specific compiler version),

 node secc-upload-archive.js --gcc /usr/bin/gcc-5 /usr/bin/g++-5 ./secc-create-archive-linux.js http://172.17.42.1:10509

 node secc-upload-archive.js --clang /usr/bin/clang /usr/bin/clang++ ./secc-create-archive-linux.js http://172.17.42.1:10509

Daemon - n PC

edit "daemon" part in 'settings.json' file. set scheduler's address and port.

if you want to use cache, go #Caches section.

run 'node secc-daemon.js' as root for chroot-jail.

$ sudo DEBUG=secc* node secc-daemon.js

Scheduler - 1 PC

$ DEBUG=secc* node secc-scheduler.js

Modes

MODE 1 - Preprocessed Mode(default) send one preprocessed source to daemon. (cause CPU load)

MODE 2 - Pump Mode send headers and source to daemon. (cause Memory/Bandwidth load)

MODE 3 - Git Mode(not yet supported.)

Debug

  • DEBUG=* to watch every verbose logs.
  • DEBUG=secc* to watch only SECC's log.
  • SECC_LOG=/path/to/log.txt - redirect logs to a file
$ DEBUG=* SECC_MODE=1 SECC_CACHE=1 /path/to/secc/bin/gcc -c test.c

Caches

install REDIS in a daemon computer. then, enable "cache" in "daemon" part of 'settings.json' file.

Environment Variable

ENV secc-client secc-daemon secc-scheduler
SECC_ADDRESS scheduler address scheduler address -
SECC_PORT scheduler port scheduler port listening port
DEBUG log level log level log level
SECC_LOG log file path log file path log file path
SECC_CMDLINE log command line - -
SECC_MODE processed or pump - -
SECC_CACHE cache prefer cache enable -
SECC_CROSS cross-compiling prefer - -
SECC_CC c compiler path - -
SECC_CXX c++ compiler path - -
REDIS_ADDRESS - redis address -
REDIS_PORT - redis port -
SECC_EXPOSE_ADDRESS - address for client -
SECC_EXPOSE_PORT - port for client -

How It Works

  • Scheduler - RESTful/WebSocket/Monitoring WebServer
  • Daemon - RESTful WebServer + WebSocket Client + Cache Server(redis)
  • Client - http client
  • (once) Client uploads a Compiler Archive to Scheduler
  • Scheduler and Daemons are connected by WebSocket
  • Client asks to Scheduler which daemon is available by REST API
  • (Optional) Client tries to get caches if possible
  • Client sends a source or/and dependencies to a Daemon by REST API
  • (once) Daemon downloads the archive from Scheduler if not exists
  • Daemon compiles the sources by Client's Compiler Archive
  • Daemon responds to Client with a object(+@)
  • (Optional) Objects are stored in Daemon's MemoryDB(redis)

License

MIT

Current Tags

  • 0.2.112                                ...           latest (3 years ago)

34 Versions

  • 0.2.112                                ...           3 years ago
  • 0.2.108                                ...           3 years ago
  • 0.2.97                                ...           3 years ago
  • 0.2.84                                ...           3 years ago
  • 0.2.71                                ...           3 years ago
  • 0.2.64                                ...           3 years ago
  • 0.2.58                                ...           3 years ago
  • 0.2.45                                ...           3 years ago
  • 0.2.36                                ...           3 years ago
  • 0.2.35                                ...           3 years ago
  • 0.2.26                                ...           4 years ago
  • 0.2.24                                ...           4 years ago
  • 0.2.22                                ...           4 years ago
  • 0.2.21                                ...           4 years ago
  • 0.2.10                                ...           4 years ago
  • 0.2.5                                ...           4 years ago
  • 0.2.0                                ...           4 years ago
  • 0.1.26                                ...           4 years ago
  • 0.1.22                                ...           4 years ago
  • 0.1.19                                ...           4 years ago
  • 0.1.18                                ...           4 years ago
  • 0.1.17                                ...           4 years ago
  • 0.1.16                                ...           4 years ago
  • 0.1.14                                ...           4 years ago
  • 0.1.13                                ...           4 years ago
  • 0.1.12                                ...           4 years ago
  • 0.1.11                                ...           4 years ago
  • 0.1.10                                ...           4 years ago
  • 0.1.9                                ...           4 years ago
  • 0.1.8                                ...           4 years ago
  • 0.1.6                                ...           4 years ago
  • 0.1.2                                ...           4 years ago
  • 0.1.1                                ...           4 years ago
  • 0.1.0                                ...           4 years ago
Maintainers (1)
Downloads
Today 0
This Week 0
This Month 1
Last Day 0
Last Week 1
Last Month 34
Dependencies (15)
Dev Dependencies (0)
None
Dependents (1)

Copyright 2014 - 2016 © taobao.org |