Gitiles
Code Review Sign In
gerrit.avm99963.com / monorail-avm99963 / f5e1039344c617e3602dcdf0a4e0aac52f50b3d8
commitf5e1039344c617e3602dcdf0a4e0aac52f50b3d8[log] [tgz]
authorAdrià Vilanova Martínez <me@avm99963.com>Tue Dec 07 22:55:40 2021 +0100
committergerritbot <gerritbot@avm99963.com>Tue Dec 07 23:38:12 2021 +0100
tree2193725f0c8126aca918efea8ce3543c66dbcf3f
parent535e73145085d616b593b2132cc989793572bf17 [diff]
Merge branch 'main' into avm99963-monorail

GitOrigin-RevId: 1412a00daa7379d93b6872e9ef7da0a45118b180
25 files changed
tree: 2193725f0c8126aca918efea8ce3543c66dbcf3f
  1. api/
  2. businesslogic/
  3. doc/
  4. features/
  5. framework/
  6. google/
  7. project/
  8. proto/
  9. schema/
  10. search/
  11. services/
  12. sitewide/
  13. static/
  14. static_src/
  15. templates/
  16. testing/
  17. third_party/
  18. tools/
  19. tracker/
  20. components ⇨ ../../luci/appengine/components/components
  21. gae.py ⇨ ../../../infra/luci/appengine/components/tools/gae.py
  22. gae_ts_mon ⇨ ../../../infra/appengine_module/gae_ts_mon
  23. testing_utils ⇨ ../../../infra/appengine_module/testing_utils/
  24. .babelrc
  25. .coveragerc
  26. .eslintrc.json
  27. .expect_tests.cfg
  28. .expect_tests_pretest.py
  29. .gitignore
  30. .prettierrc.json
  31. .style.yapf
  32. .testcoveragerc
  33. app.yaml.m4
  34. appengine_config.py
  35. codereview.settings
  36. cron.yaml
  37. dev-services.yml
  38. jsconfig.json
  39. karma.conf.js
  40. Makefile
  41. module-api.yaml.m4
  42. module-besearch.yaml.m4
  43. module-latency-insensitive.yaml.m4
  44. monorailapp.py
  45. OWNERS
  46. package-lock.json
  47. package.json
  48. PRESUBMIT.py
  49. queue.yaml
  50. README.md
  51. registerpages.py
  52. requirements.dev.txt
  53. requirements.py2.txt
  54. requirements.txt
  55. settings.py
  56. tsconfig.json
  57. webpack.config.js
README.md

Monorail Issue Tracker - bugs.avm99963.com

This repo is a fork of the open-source Monorail issue tracker designed to run at bugs.avm99963.com. I've made the code public so it can inspire other developers to host relatively cheap Monorail instances.

The original README.md file is here below:

Monorail Issue Tracker

Monorail is the Issue Tracker used by the Chromium project and other related projects. It is hosted at bugs.chromium.org.

If you wish to file a bug against Monorail itself, please do so in our self-hosting tracker. We also discuss development of Monorail at infra-dev@chromium.org.

Getting started with Monorail development

For Googlers: Monorail's codebase is open source and can be installed locally on your workstation of choice.

Here's how to run Monorail locally for development on MacOS and Debian stretch/buster or its derivatives.

  1. You need to get the Chrome Infra depot_tools commands to check out the source code and all its related dependencies and to be able to send changes for review.

  2. Check out the Monorail source code

    1. cd /path/to/empty/workdir
    2. fetch infra
    3. cd infra/appengine/monorail

  3. Make sure you have the AppEngine SDK:

    1. It should be fetched for you by step 1 above (during runhooks)
    2. Otherwise, you can download it from https://developers.google.com/appengine/downloads#Google_App_Engine_SDK_for_Python

  4. Spin up dependent services.

    1. We use docker and docker-compose to orchestrate. So install docker and docker-compose first. For glinux users see go/docker
    2. Make sure to authenticate with the App Engine SDK and configure Docker. This is needed to install Cloud Tasks Emulator.
      1. gcloud auth login
      2. gcloud auth configure-docker
        1. If you get authentication errors here, run setenv PATH = $PATH;/path/to/gcloud/bin
    3. Run docker-compose -f dev-services.yml up -d. This should spin up:
      1. MySQL v5.6
      2. Redis
      3. Cloud Tasks Emulator
        1. TODO host this publicly and remove section.
        2. This will require you to authenticate to Google Container Registry to pull the docker image: gcloud auth login gcloud auth configure-docker. If you're an open source developer and do not have access to the monorail project and thereby its container registry you will need to start the Cloud Tasks Emulator from source

  5. Set up SQL database. (You can keep the same sharding options in settings.py that you have configured for production.).

    1. Copy setup schema into the docker container
      1. docker cp schema/. mysql:/schema
      2. docker exec -it mysql bash
      3. mysql --user=root monorail < schema/framework.sql
      4. mysql --user=root monorail < schema/project.sql
      5. mysql --user=root monorail < schema/tracker.sql
      6. exit

  6. Configure the site defaults in settings.py. You can leave it as-is for now.

  7. Set up the front-end development environment:

    1. On Debian
      1. eval `../../go/env.py` -- you'll need to run this in any shell you wish to use for developing Monorail. It will add some key directories to your $PATH.
      2. Install build requirements:
        1. sudo apt-get install build-essential automake
    2. On MacOS
      1. Install homebrew
      2. Install node and npm

        1. Install node version manager brew install nvm

        2. See the brew instructions on updating your shell's configuration

        3. Install node and npm nvm install 12.13.0

        4. Add the following to the end of your ~/.zshrc file:

          export NVM_DIR="$HOME/.nvm"
[ -s "/usr/local/opt/nvm/nvm.sh" ] && . "/usr/local/opt/nvm/nvm.sh"  # This loads nvm
[ -s "/usr/local/opt/nvm/etc/bash_completion.d/nvm" ] && . "/usr/local/opt/nvm/etc/bash_completion.d/nvm"  # This loads nvm bash_completion

  8. Install Python and JS dependencies:

    1. Install MySQL, needed for mysqlclient
      1. For mac: brew install mysql@5.6
      2. For Debian derivatives, download and unpack this bundle: tar -xf mysql-server_5.6.40-1ubuntu14.04_amd64.deb-bundle.tar. Install the packages in the order of mysql-common,mysql-community-client, mysql-client, then mysql-community-server.
    2. Optional: You may need to install pip. You can verify whether you have it installed with which pip.
      1. curl -O https://bootstrap.pypa.io/2.7/get-pip.py
      2. sudo python get-pip.py
    3. Use virtualenv to keep from modifying system dependencies.
      1. sudo pip install virtualenv
      2. virtualenv venv to set up virtualenv within your monorail directory.
      3. source venv/bin/activate to activate it, needed in each terminal instance of the directory.
    4. Mac only: install libssl, needed for mysqlclient.
      1. brew install openssl; export LIBRARY_PATH=$LIBRARY_PATH:/usr/local/opt/openssl/lib/
    5. make dev_deps
    6. make deps

  9. Run the app:

    1. make serve

  10. Browse the app at localhost:8080 your browser.

  11. Set up your test user account (these steps are a little odd, but just roll with it): 1. Sign in using test@example.com 1. Log back out and log in again as example@example.com 1. Log out and finally log in again as test@example.com. 1. Everything should work fine now.

  12. Modify your Monorail User row in the database and make that user a site admin. You will need to be a site admin to gain access to create projects through the UI.

    1. docker exec mysql mysql --user=root monorail -e "UPDATE User SET is_site_admin = TRUE WHERE email = 'test@example.com';"
    2. If the admin change isn't immediately apparent, you may need to restart your local dev appserver. If you kill the dev server before running the docker command, the restart may not be necessary.

Instructions for deploying Monorail to an existing instance or setting up a new instance are here.

Feature Launch Tracking

To set up FLT/Approvals in Monorail:

  1. Visit the gear > Development Process > Labels and fields
  2. Add at least one custom field with type "Approval" (this will be your approval
  3. Visit gear > Development Process > Templates
  4. Check "Include Gates and Approval Tasks in issue"
  5. Fill out the chart - The top row is the gates/phases on your FLT issue and you can select radio buttons for which gate each approval goes

Testing

To run all Python unit tests, in the appengine/monorail directory run:

make test

For quick debugging, if you need to run just one test you can do the following. For instance for the test IssueServiceTest.testUpdateIssues_Normal in services/test/issue_svc_test.py:

../../test.py test appengine/monorail:services.test.issue_svc_test.IssueServiceTest.testUpdateIssues_Normal --no-coverage

Frontend testing

To run the frontend tests for Monorail, you first need to set up your Go environment. From the Monorail directory, run:

eval `../../go/env.py`

Then, to run the frontend tests, run:

make karma

If you want to skip the coverage for karma, run:

make karma_debug

To run only one test or a subset of tests, you can add .only to the test function you want to isolate:

// Run one test.
it.only(() => {
  ...
});

// Run a subset of tests.
describe.only(() => {
  ...
});

Just remember to remove them before you upload your CL.

Troubleshooting

  • BindError: Unable to bind localhost:8080

This error occurs when port 8080 is already being used by an existing process. Oftentimes, this is a leftover Monorail devserver process from a past run. To quit whatever process is on port 8080, you can run kill $(lsof -ti:8080).

  • RuntimeError: maximum recursion depth exceeded while calling a Python object

If running make serve gives an output similar to this, make sure you're using a virtual environment (see above for how to configure one). Then, make the changes outlined in this CL.

  • gcloud: command not found

Add the following to your ~/.zshrc file: alias gcloud='/Users/username/google-cloud-sdk/bin/gcloud'. Replace username with your Google username.

  • TypeError: connect() got an unexpected keyword argument 'charset'

This error occurs when dev_appserver cannot find the MySQLdb library. Try installing it via sudo apt-get install python-mysqldb.

  • TypeError: connect() argument 6 must be string, not None

This occurs when your mysql server is not running. Check if it is running with ps aux | grep mysqld. Start it up with /etc/init.d/mysqld start on linux, or just mysqld.

  • dev_appserver says OSError: [Errno 24] Too many open files and then lists out all source files

dev_appserver wants to reload source files that you have changed in the editor, however that feature does not seem to work well with multiple GAE modules and instances running in different processes. The workaround is to control-C or kill the dev_appserver processes and restart them.

  • IntegrityError: (1364, "Field 'comment_id' doesn't have a default value") happens when trying to file or update an issue

In some versions of SQL, the STRICT_TRANS_TABLES option is set by default. You'll have to disable this option to stop this error.

  • ImportError: No module named six.moves

You may not have six.moves in your virtual environment and you may need to install it.

  1. Determine that python and pip versions are possibly in vpython-root
    1. which python
    2. which pip
  2. If your python and pip are in vpython-root
    1. sudo `which python` `which pip` install six

Development resources

Supported browsers

Monorail supports all browsers defined in the Chrome Ops guidelines.

File a browser compatability bug here.

Frontend code practices

See: Monorail Frontend Code Practices

Monorail's design

Triage process

See: Monorail Triage Guide.

Release process

See: Monorail Deployment

User guide

For information on how to use Monorail, see the Monorail User Guide.

Setting up a new instance of Monorail

See: Creating a new Monorail instance