Quckoo is a fault-tolerant distributed task scheduler platform that runs on the JVM. The aim of the project is the implementation of a reliable system able to run large amount of scheduled tasks without single points of failure.
To achieve that, Quckoo is composed of a cluster of scheduler nodes and ad hoc worker nodes that connect to this cluster and request for work to be sent to them. It's basically a generalization of the distributed worker pattern (and in fact the implementation owes a lot to the previous post).
First of all, ensure you have the following tools in your local machine:
- Java JDK 1.8+ for running and compiling the source code
- SBT 0.13.5+ for building the whole project
- NodeJS 4.5.x+ for running and optimizing the ScalaJS code
- PhantomJS 2.x+ for running the ScalaJS test suites
- SASS 3.4.0+ for compiling UI's stylesheets.
Optionally, if you will want to be able to run Quckoo in your local machine using the Docker containers you will need one of the following (or both):
- Docker 1.10+ for running the docker containers "natively" in your local machine
- Vagrant 1.8.x+ plus a Virtual Machine provider (i.e.: VirtualBox 5.x+) for bootstrapping the sandbox.
To build the project input following commands in a terminal:
git clone https://github.com/alonsodomin/quckoo
cd quckoo
sbt package
The easiest way to launch the cluster right now is by building the Docker images locally and then starting the default docker compose cluster setup. This operations can easily be done from within SBT:
sbt launchLocal
After the previous commands, using docker ps you should be able to see 2 master nodes, one worker node and one Cassandra node running as docker containers
in your local machine. The cluster will need a few seconds to fully stabilize and then it will be ready to serve requests. You should be able to
access the Quckoo UI in following URL:
Use admin and password as credentials when prompted.
The repository ships with a Vagrantfile in its root folder for ease the setup of a fully isolated environment. This Vagrant configuration
will create a virtual machine in the host computer and deploy into it the required components of the architecture as
Docker containers. To start this environment issue following command in a terminal window from inside your working copy:
vagrant up
To access the UI in this environment, use this URL: http://192.168.50.25:8095
When trying to bootstrap the self-contained VM with Vagrant, SBT will be invoked from the inside the VM to publish the Docker images in the host's local registry. When this is done the first time SBT may fail to communicate with the local Docker daemon and you will see a build error. If this happens just re-provision the VM and it should work.
vagrant provision
Quckoo is still right now in experimental phase, current codebase will be evolving until it reaches the level of robustness necessary to be able to trust on the system to be able to handle the big load we are aiming for. Feel free to fork this repository and contribute with your knowledge to make this project a reliable and really fault-tolerant task scheduling platform.
Copyright 2015 Antonio Alonso Dominguez
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.