movement-data-toolkit
Tools and utilities for working with Uber Movement data
Last updated 7 hours ago by benmkramer .
MIT · Bugs · Original npm · Tarball · package.json
$ cnpm install movement-data-toolkit 
SYNC missed versions from official npm registry.

Tools and utilities for working with Uber Movement data

oclif Version License


???? Getting Started

Using this toolkit requires Node.js to be installed on your machine, see Downloading and installing Node.js and npm. We recommend using nvm as described in that document, or brew install node for homebrew users. Using the official GUI installer from nodejs.org may lead to permissions issues when installing new commands.

Once node is installed, you can install and run the toolkit as shown in the sections below.

???? Usage

$ npm install -g movement-data-toolkit
$ mdt COMMAND
running command...
$ mdt (-v|--version|version)
movement-data-toolkit/1.2.7 darwin-x64 node-v8.16.2
$ mdt --help [COMMAND]
USAGE
  $ mdt COMMAND
...

???? Commands

mdt clear-cache

Some commands in this toolkit download files to a temporary directory. These files can be rather large, and it can be helpful to clear this data to reclaim disk space.

USAGE
  $ mdt clear-cache

EXAMPLE
  $ mdt clear-cache

mdt create-geometry-file [CITY] [YEAR]

Creates a geojson file that contains the geometries for each road segment in the specified city, as well as both the corresponding OSM identifiers for each road segment.

USAGE
  $ mdt create-geometry-file [CITY] [YEAR]

ARGUMENTS
  CITY  Slug of the city to download data for. You can find this value in the URL when accessing data on the Movement
        website.

  YEAR  Movement publishes mapping files and road geometries yearly, specify the year you would like to use here

OPTIONS
  -o, --output=output        File to write output to (writes to stdout by default)

  --bbox=bbox                Features outside of this bounding box will be removed from output. Should be of the form
                             minX,minY,maxX,maxY like "-10.0,-10.0,10.0,10.0" (see
                             https://tools.ietf.org/html/rfc7946#section-5)

  --driving-side=LEFT|RIGHT  [default: RIGHT] Road geometries are "twinned" in the output file to visualize speeds in
                             both directions, to do this accurately you should use --driving-side="LEFT" for places like
                             the UK or Australia.

  --format=GEOJSON|CSV       [default: GEOJSON] Format to use for the output file

  --geom-format=GEOJSON|WKT  [default: GEOJSON] Changes format of geometries in output file (only supported with
                             --format=CSV)

  --log-level=INFO|DEBUG     [default: INFO] What types of logs should be emitted?

  --[no-]twin                Whether or not to "twin" road geometries in the output file – this duplicates and offset
                             two-way roads so you can visualize data in both directions. Defaults to true.

DESCRIPTION
  The produced file can help avoid the need to download and manipulate OSM geometries from the web yourself.

EXAMPLE
  $ mdt create-geometry-file cincinnati 2018 > my-output-file.geojson

mdt help [COMMAND]

display help for mdt

USAGE
  $ mdt help [COMMAND]

ARGUMENTS
  COMMAND  command to show help for

OPTIONS
  --all  see all commands in CLI

See code: @oclif/plugin-help

mdt speeds-to-geojson [CITY] [STARTDATE] [ENDDATE]

Downloads and processes Movement speeds data into a geojson file that can be visualized with Kepler.gl or other geospatial visualization tools.

USAGE
  $ mdt speeds-to-geojson [CITY] [STARTDATE] [ENDDATE]

ARGUMENTS
  CITY       Slug of the city to download data for. You can find this value in the URL when accessing data on the
             Movement website.

  STARTDATE  Fetch data starting from this date

  ENDDATE    Fetch data up to and including this date

OPTIONS
  -o, --output=output        File to write output to (writes to stdout by default)

  --bbox=bbox                Features outside of this bounding box will be removed from output. Should be of the form
                             minX,minY,maxX,maxY like "-10.0,-10.0,10.0,10.0" (see
                             https://tools.ietf.org/html/rfc7946#section-5)

  --driving-side=LEFT|RIGHT  [default: RIGHT] Road geometries are "twinned" in the output file to visualize speeds in
                             both directions, to do this accurately you should use --driving-side="LEFT" for places like
                             the UK or Australia.

  --log-level=INFO|DEBUG     [default: INFO] What types of logs should be emitted?

  --[no-]twin                Whether or not to "twin" road geometries in the output file – this duplicates and offset
                             two-way roads so you can visualize data in both directions. Defaults to true.

DESCRIPTION
  Note: Becuase our speeds data CSVs have an hourly granularity, the results produced by this command are technically an 
  average of averages and may differ slightly from results shown in the app at movement.uber.com.

EXAMPLE
  $ mdt speeds-to-geojson cincinnati 2018-01-01 2018-01-31 > my-output-file.geojson

mdt speeds-transform [TYPE] [CITY] [STARTDATE] [ENDDATE]

Downloads and transforms Movement Speeds data, filtering by date range and optional bounding box and producing a CSV with speed values mapped to OSM identifiers.

USAGE
  $ mdt speeds-transform [TYPE] [CITY] [STARTDATE] [ENDDATE]

ARGUMENTS
  TYPE       Either "historical" for the monthly historical time series or "stats" for the quarterly statistics file.

  CITY       Slug of the city to download data for. You can find this value in the URL when accessing data on the
             Movement website.

  STARTDATE  Fetch data starting from this date

  ENDDATE    Fetch data up to and including this date

OPTIONS
  -o, --output=output     File to write output to (writes to stdout by default)

  --bbox=bbox             Features outside of this bounding box will be removed from output. Should be of the form
                          minX,minY,maxX,maxY like "-10.0,-10.0,10.0,10.0" (see
                          https://tools.ietf.org/html/rfc7946#section-5)

  --log-level=INFO|DEBUG  [default: INFO] What types of logs should be emitted?

  --[no-]osm-ids          Whether or not the output file should use OSM IDs, defaults to true. If false, will use Uber
                          Movement IDs.

EXAMPLE
  $ mdt speeds-transform historical cincinnati 2018-01-01 2018-01-31 > filtered.csv

???? License

The code in this package is licensed under the MIT License, see LICENSE file at the root of this project.

Data specific to Uber Movement – including speeds or travel times – is licensed under the Creative Commons, Attribution Non Commercial License

Data from OpenStreetMap – including road geometries or other OSM metadata and identifiers – is licensed under the Open Data Commons Open Database License (ODbL) by the OpenStreetMap Foundation (OSMF).

⚠️ Troubleshooting

Out of memory Errors (OOM)

Occasionally you may run into an error that looks something like this:

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

Some of the scripts in this toolkit are processing a lot of data. By default Node.js uses 512mb, but we can increase that limit with another command line flag:

export NODE_OPTIONS="--max_old_space_size=8192" && mdt [command]

The above command would allow the node process to use up to 8gb of memory while it's running.

EACCES permissions errors when installing

The default Node installer from nodejs.org installs Node in a way that can cause permissions issues when you run npm packages globally.

You can either re-install node using a node version manager or change npms global package directory to one you have access to.

???? Issues & Feedback

Questions? Issues? Feedback? Let us know on Movement's Contact Form.

Current Tags

  • 1.2.7                                ...           latest (7 hours ago)

21 Versions

  • 1.2.7                                ...           7 hours ago
  • 1.2.5                                ...           4 months ago
  • 1.2.4                                ...           4 months ago
  • 1.2.3                                ...           4 months ago
  • 1.2.2                                ...           4 months ago
  • 1.2.1                                ...           4 months ago
  • 1.2.0                                ...           4 months ago
  • 1.1.5                                ...           5 months ago
  • 1.1.4                                ...           5 months ago
  • 1.1.3                                ...           5 months ago
  • 1.1.2                                ...           5 months ago
  • 1.0.0                                ...           5 months ago
  • 0.0.9                                ...           5 months ago
  • 0.0.8                                ...           5 months ago
  • 0.0.7                                ...           5 months ago
  • 0.0.6                                ...           5 months ago
  • 0.0.5                                ...           5 months ago
  • 0.0.4                                ...           5 months ago
  • 0.0.3                                ...           5 months ago
  • 0.0.2                                ...           5 months ago
  • 0.0.1                                ...           5 months ago
Maintainers (1)
Downloads
Today 13
This Week 13
This Month 13
Last Day 0
Last Week 0
Last Month 2
Dependencies (18)
Dev Dependencies (22)
Dependents (0)
None

Copyright 2014 - 2016 © taobao.org |