Gitiles
Code Review Sign In
gerrit.avm99963.com / monorail-avm99963 / 515639b5db799e6c895a5c304f6d975ec6fd36ea
commit515639b5db799e6c895a5c304f6d975ec6fd36ea[log] [tgz]
authorAdrià Vilanova Martínez <me@avm99963.com>Tue Jul 06 16:43:59 2021 +0200
committergerritbot <gerritbot@avm99963.com>Sun Oct 17 01:01:55 2021 +0200
treebe082ed9b4127a98f550dd5b62d659db21983725
parent854996b0b589716b61d700069c8d23e4920787e7 [diff]
First customization to avm99963-bugs

- Only a service is used (default) in order to keep usage within Google
  Cloud's free tier.
- Settings have been adapted.
- Only a single DB instance is used (primary).
- Some cron jobs are executed less often.
- |threadsafe| has been set to "yes" so the single instance can serve
  concurrent requests (otherwise the search/list issues function
  breaks).

GitOrigin-RevId: 52130750a30a5aa2234a6dd3f5ae23c3fb1b8343
9 files changed
tree: be082ed9b4127a98f550dd5b62d659db21983725
  1. api/
  2. businesslogic/
  3. doc/
  4. features/
  5. framework/
  6. google/
  7. project/
  8. proto/
  9. schema/
  10. search/
  11. services/
  12. sitewide/
  13. static/
  14. static_src/
  15. templates/
  16. testing/
  17. third_party/
  18. tools/
  19. tracker/
  20. components ⇨ ../../luci/appengine/components/components
  21. gae.py ⇨ ../../../infra/luci/appengine/components/tools/gae.py
  22. gae_ts_mon ⇨ ../../../infra/appengine_module/gae_ts_mon
  23. testing_utils ⇨ ../../../infra/appengine_module/testing_utils/
  24. .babelrc
  25. .coveragerc
  26. .eslintrc.json
  27. .expect_tests.cfg
  28. .expect_tests_pretest.py
  29. .gitignore
  30. .prettierrc.json
  31. .style.yapf
  32. .testcoveragerc
  33. app.yaml.m4
  34. appengine_config.py
  35. codereview.settings
  36. cron.yaml
  37. dev-services.yml
  38. jsconfig.json
  39. karma.conf.js
  40. Makefile
  41. module-api.yaml.m4
  42. module-besearch.yaml.m4
  43. module-latency-insensitive.yaml.m4
  44. monorailapp.py
  45. OWNERS
  46. package-lock.json
  47. package.json
  48. PRESUBMIT.py
  49. queue.yaml
  50. README.md
  51. registerpages.py
  52. requirements.dev.txt
  53. requirements.py2.txt
  54. requirements.txt
  55. settings.py
  56. tsconfig.json
  57. webpack.config.js
README.md

Monorail Issue Tracker

Monorail is the Issue Tracker used by the Chromium project and other related projects. It is hosted at bugs.chromium.org.

If you wish to file a bug against Monorail itself, please do so in our self-hosting tracker. We also discuss development of Monorail at infra-dev@chromium.org.

Getting started with Monorail development

For Googlers: Monorail's codebase is open source and can be installed locally on your workstation of choice.

Here's how to run Monorail locally for development on MacOS and Debian stretch/buster or its derivatives.

  1. You need to get the Chrome Infra depot_tools commands to check out the source code and all its related dependencies and to be able to send changes for review.
  2. Check out the Monorail source code
    1. cd /path/to/empty/workdir
    2. fetch infra
    3. cd infra/appengine/monorail
  3. Make sure you have the AppEngine SDK:
    1. It should be fetched for you by step 1 above (during runhooks)
    2. Otherwise, you can download it from https://developers.google.com/appengine/downloads#Google_App_Engine_SDK_for_Python
  4. Spin up dependent services.
    1. We use docker and docker-compose to orchestrate. So install docker and docker-compose first. For glinux users see go/docker
    2. Make sure to authenticate with the App Engine SDK and configure Docker. This is needed to install Cloud Tasks Emulator.
      1. gcloud auth login
      2. gcloud auth configure-docker
    3. Run docker-compose -f dev-services.yml up -d. This should spin up:
      1. MySQL v5.6
      2. Redis
      3. Cloud Tasks Emulator
        1. TODO host this publicly and remove section.
        2. This will require you to authenticate to Google Container Registry to pull the docker image: gcloud auth login gcloud auth configure-docker. If you're an open source developer and do not have access to the monorail project and thereby its container registry you will need to start the Cloud Tasks Emulator from source
  5. Set up SQL database. (You can keep the same sharding options in settings.py that you have configured for production.).
    1. Copy setup schema into the docker container
      1. docker cp schema/. mysql:/schema
      2. docker exec -it mysql bash
      3. mysql --user=root monorail < schema/framework.sql
      4. mysql --user=root monorail < schema/project.sql
      5. mysql --user=root monorail < schema/tracker.sql
      6. exit
  6. Configure the site defaults in settings.py. You can leave it as-is for now.
  7. Set up the front-end development environment:
    1. On Debian
      1. eval `../../go/env.py` -- you'll need to run this in any shell you wish to use for developing Monorail. It will add some key directories to your $PATH.
      2. Install build requirements:
        1. sudo apt-get install build-essential automake
    2. On MacOS
      1. Install node and npm
        1. Install node version manager brew install nvm
        2. See the brew instructions on updating your shell's configuration
        3. Install node and npm nvm install 12.13.0
  8. Install Python and JS dependencies:
    1. Install MySQL, needed for mysqlclient
      1. For mac: brew install mysql@5.6
      2. For Debian derivatives, download and unpack this bundle: tar -xf mysql-server_5.6.40-1ubuntu14.04_amd64.deb-bundle.tar. Install the packages in the order of mysql-common,mysql-community-client, mysql-client, then mysql-community-server.
    2. Optional: You may need to install pip. You can verify whether you have it installed with which pip.
      1. curl -O https://bootstrap.pypa.io/2.7/get-pip.py
      2. sudo python get-pip.py
    3. Optional: Use virtualenv to keep from modifying system dependencies.
      1. sudo pip install virtualenv
      2. virtualenv venv to set up virtualenv within your monorail directory.
      3. source venv/bin/activate to activate it, needed in each terminal instance of the directory.
    4. Mac only: install libssl, needed for mysqlclient.
      1. brew install openssl; export LIBRARY_PATH=$LIBRARY_PATH:/usr/local/opt/openssl/lib/
    5. make dev_deps
    6. make deps
  9. Run the app:
    1. make serve
  10. Browse the app at localhost:8080 your browser.
  11. Optional: Create/modify your Monorail User row in the database and make that user a site admin. You will need to be a site admin to gain access to create projects through the UI.
    1. docker exec mysql mysql --user=root monorail -e "UPDATE User SET is_site_admin = TRUE WHERE email = 'test@example.com';"
    2. If the admin change isn't immediately apparent, you may need to restart your local dev appserver.

Instructions for deploying Monorail to an existing instance or setting up a new instance are here.

Here's how to run unit tests from the command-line:

Testing

To run all Python unit tests, in the appengine/monorail directory run:

make test

For quick debugging, if you need to run just one test you can do the following. For instance for the test IssueServiceTest.testUpdateIssues_Normal in services/test/issue_svc_test.py:

../../test.py test appengine/monorail:services.test.issue_svc_test.IssueServiceTest.testUpdateIssues_Normal --no-coverage

Frontend testing

To run the frontend tests for Monorail, you first need to set up your Go environment. From the Monorail directory, run:

eval `../../go/env.py`

Then, to run the frontend tests, run:

make karma

If you want to skip the coverage for karma, run:

make karma_debug

To run only one test or a subset of tests, you can add .only to the test function you want to isolate:

// Run one test.
it.only(() => {
  ...
});

// Run a subset of tests.
describe.only(() => {
  ...
});

Just remember to remove them before you upload your CL.

Troubleshooting

  • BindError: Unable to bind localhost:8080

This error occurs when port 8080 is already being used by an existing process. Oftentimes, this is a leftover Monorail devserver process from a past run. To quit whatever process is on port 8080, you can run kill $(lsof -ti:8080).

  • TypeError: connect() got an unexpected keyword argument 'charset'

This error occurs when dev_appserver cannot find the MySQLdb library. Try installing it via sudo apt-get install python-mysqldb.

  • TypeError: connect() argument 6 must be string, not None

This occurs when your mysql server is not running. Check if it is running with ps aux | grep mysqld. Start it up with /etc/init.d/mysqld start on linux, or just mysqld.

  • dev_appserver says OSError: [Errno 24] Too many open files and then lists out all source files

dev_appserver wants to reload source files that you have changed in the editor, however that feature does not seem to work well with multiple GAE modules and instances running in different processes. The workaround is to control-C or kill the dev_appserver processes and restart them.

  • IntegrityError: (1364, "Field 'comment_id' doesn't have a default value") happens when trying to file or update an issue

In some versions of SQL, the STRICT_TRANS_TABLES option is set by default. You'll have to disable this option to stop this error.

  • ImportError: No module named six.moves

You may not have six.moves in your virtual environment and you may need to install it.

  1. Determine that python and pip versions are possibly in vpython-root
    1. which python
    2. which pip
  2. If your python and pip are in vpython-root
    1. sudo `which python` `which pip` install six

Development resources

Supported browsers

Monorail supports all browsers defined in the Chrome Ops guidelines.

File a browser compatability bug here.

Frontend code practices

See: Monorail Frontend Code Practices

Monorail's design

Triage process

See: Monorail Triage Guide.

Release process

See: Monorail Deployment

User guide

For information on how to use Monorail, see the Monorail User Guide.

Setting up a new instance of Monorail

See: Creating a new Monorail instance