Contributing to Changes
=======================
To get started, either get a job at Dropbox, or sign our CLA:
https://opensource.dropbox.com/cla/
Setup
-----
Start by configuring your environment per the :doc:`setup guide `. You can safely skip the various production requirements (such as setting up services).
Webserver
~~~~~~~~~
Run the webserver:
.. code-block:: bash
$ bin/web
.. note:: The server doesn't automatically reload when you make changes to the Python code.
Background Workers
~~~~~~~~~~~~~~~~~~
While it's likely you won't need to actually run the workers, they're managed via `Celery `_.
.. code-block:: bash
# Start a generic worker process
# the -B flag indicates to also start "celerybeat" which
# is utilized for periodic tasks.
$ bin/worker -B
.. note:: In development you can set ``CELERY_ALWAYS_EAGER=True`` to run the queue tasks synchronously in-process. Generally we prefer to test through automated integration tests, but this is useful if you want to QA and don't want to run several processes.
Directory Layout
----------------
While there are a significant and growing number of paths, this is an attempt to outline some of the more common and important code paths.
.. code-block:: bash
# command line scripts
├── bin
# python code
├── changes
# the core of url registration and app configuration
│ ├── config.py
# api controllers and serializers
│ ├── api
# various integration code (primarily legacy for communicating with Jenkins)
│ ├── backends
# implementations of the various buildsteps (modern build handlers)
│ ├── buildsteps
# database utilities
│ ├── db
# implementations of build factory expanders
│ ├── expanders
# tasks executed asynchronously via Celery workers
│ ├── jobs
# our sqlalchemy model definitions
│ ├── models
# integration code for mercurial/git
│ └── vcs
# python test bootstrap code
├── conftest.py
# docs, like what you're reading right now
├── docs
# database migrations (via Alembic)
├── migrations
# client-side templates
├── partials
# static media (such as the frontend code, as well as vendored code within)
├── static
│ ├── css
│ ├── js
│ └── vendor
# server-side templates
├── templates
# all tests (only python currently)
└── tests
Understanding the Frontend
--------------------------
Everything is bundled into a "state". A state is a combination of a router and a controller, and it contains nearly all of the logic for rendering an individual page.
States are registered into `routes.js `_ (they get required and then registered to a unique name).
As an example, let's take a look at `planList.js `_,
a fairly simple state:
.. code-block:: javascript
// static/js/states/planList.js
define(['app'], function(app) {
'use strict';
return {
// parent is used for template/scope inheritance
parent: 'layout',
// the url **relative** to the parent
// in our case, layout is the parent which has no base url
url: '/plans/',
// all templates exist in partials/
templateUrl: 'partials/plan-list.html',
// $scope, planList, and Collection are all dependencies, implicitly
// parsed by angular and included in the function's scope
controller: function($scope, planList, Collection) {
// binding to $scope adds it to the template context
$scope.plans = new Collection(planList);
},
// resolvers get executed **before** the controller is run and
// are ideal for loading initial data
resolve: {
planList: function($http) {
// this **must** return a future
return $http.get('/api/0/plans/').then(function(response){
return response.data;
});
}
}
};
});
Then within `routes.js `_,
we register this under the 'plan_list' namespace:
.. code-block:: javascript
// static/js/routes.js
define([
'app',
'states/layout',
// ...
'states/planList'
], function(
// the order of dependencies must match above
app,
LayoutState,
// ...
PlanListState
) {
// this has been simplified for illustration purposes
app.config(function($stateProvider) {
$stateProvider
.state('layout', LayoutState)
// ...
.state('plan_list', PlanListState);
});
Let's take a look at the template, `plan-list.html `_:
.. code-block:: html
There's a few key things to understand in this simple example:
.. code-block:: html
The ui-view attribute here is what Angular calls a directive. In this case, it actually maps to the library we use (ui-router) and says "content within this can be replaced by the child template". That's not precisely the meaning, but for our examples it's close enough.
Jumping down to actual rendering:
.. code-block:: html
This is another built-in directive, and it says "expand 'plans', and assign the item at the current index to 'plan'".
We can then reference it:
.. code-block:: html
{{plan.name}} |
Two things are happening here:
- We're specifying ui-sref, which is saying "find the named url with these parameters". Parameters are always inherited, so you only need to pass in the changed values.
- In our specific example, we're referring to the ``plan_details`` state, which might be a child page of ``plan_list``. This is the same name you would define in the ``.state()`` registration.
- We also need to pass the ``plan_id`` parameter, which is used by the state's url matcher, and then made available via ``$stateParams`` within it's controller.
- Render the ``name`` attribute of this plan.
There's also a couple uses of our `timeSince.js `_ directive:
.. code-block:: html
|
In most uses of directives, you'll notice that we don't surround the value with ``{{ }}``. This is because the
directive itself is choosing to evaluate the value as part of the scope.
Understanding the Backend
-------------------------
The backend is a fairly straightforward Flask app. It has two primary models: task execution and consumer API.
We're not going to explain the workers as they contain a very large amount of coordination logic, but instead let's focus on the API.
To start with, the entry point for URLs currently lives in ``config.py``, under ``configure_api_routes``. You'll see that each API controller lives in a separate module space and is registered into the routing here.
Let's take a look at the API controller for our ``plan_list`` state, contained in
`plan_index.py `_:
.. code-block:: python
# changes/api/plan_index.py
from __future__ import absolute_import, division, unicode_literals
from changes.api.base import APIView
from changes.models import Plan
class PlanIndexAPIView(APIView):
def get(self):
results = Plan.query.order_by(Plan.label.asc())[:10]
# while respond() can serialize for you, we use this for illustration
# purposes
data = self.serialize(results)
return self.respond(data, serialize=False)
There's no real surprises here if you've ever written Python. We're using SQLAlchemy to query the ``Plan`` table, and we're returning a simple result of ten plans.
There are two things happening here:
- We're serializing the list of Plans using the default registered serializer (dig
into the `serializer `_ to see what this does.)
- ``respond()`` is then going to return an HTTP response object, with a 200 status code
any required headers, as well as eventually encode our Python object into JSON.
And of course, we absolutely require integration tests for every endpoint, which live
in `test_plan_index.py `_:
.. code-block:: python
from changes.testutils import APITestCase
class PlanIndexTest(APITestCase):
path = '/api/0/plans/'
def test_simple(self):
plan1 = self.plan
plan2 = self.create_plan(label='Bar')
resp = self.client.get(self.path)
assert resp.status_code == 200
data = self.unserialize(resp)
assert len(data) == 2
assert data[0]['id'] == plan2.id.hex
assert data[1]['id'] == plan1.id.hex
A ``client`` attribute exists on the test instance, as well as a number of helpers in `changes.testutils.fixtures `_ for creating mock data. This is a real database transaction so you can do just about everything, and we'll safely ensure things are cleaned up and fast.
Loading in Mock Data
--------------------
If you're changing the frontend, it's likely you're going to want some data to work with. We've provided a helper script which will create some sample data, as well as stream in continuous updates. It's not quite the same as production, but it should be enough to work with:
.. code-block:: bash
$ python stream_data.py