aviation-scraper
Scraper to get airlines and airport data
Last updated 2 years ago by cristobal-io .
MIT · Repository · Bugs · Original npm · Tarball · package.json
$ cnpm install aviation-scraper 
SYNC missed versions from official npm registry.

Codeship Status for cristobal-io/aviation-scraper Coverage Status

Aviation Scraper

This project is going to be composed of few packages that are going to be scraping from wikipedia the following information:

  • World Airports
  • World Airlines
  • World destinations

This is intended for use of an updated database of information for a map application.

Testing

Before testing this application you have to run make update-local-pages to download a local copy of the html pages required for tests.

Then run tests with make test

Usage

From the CLI

aviation-scraper -h

displays this help:

  Usage: aviation-scraper [options]

  Retrieve airlines destinations and airports data.
  Multiple options are not allowed, only '-b' and '-s' options can be combined with the rest.

  Example for saving individual files to airports:
  $ aviation-scraper -a -s true

  Example for saving file to an specific directory:
  $./aviation-scraper -a -b './my_new_directory'

  Options:

  -h, --help            output usage information
  -V, --version         output the version number
  -l, --list            List of airports with the link to the wikipedia page. (Saved to a single file)
  -d, --destinations    Destinations of all the airlines listed on the wikipedia. (saved each airline with an individual JSON file)
  -a, --airports        Saves the important data for each airport (saved in a single JSON file and optional for each airport)
  -c, --companies       Saves all the airline links and all the important information for each airline.
  -b, --base <basedir>  the base directory where to save the files generated (default: tmp)
  -s, --save <save>     this options allows you to save individual files for each airline, airport or destination (default:false)

Contribute

if you wish to contribute create your branch and create a pull request on the dev branch or you can create an issue.

Current Tags

  • 0.2.4                                ...           latest (2 years ago)

7 Versions

  • 0.2.4                                ...           2 years ago
  • 0.2.3                                ...           3 years ago
  • 0.2.1                                ...           3 years ago
  • 0.2.0                                ...           3 years ago
  • 0.1.0                                ...           3 years ago
  • 0.0.13                                ...           3 years ago
  • 0.0.12                                ...           3 years ago
Maintainers (1)
Downloads
Today 0
This Week 0
This Month 1
Last Day 0
Last Week 1
Last Month 0
Dependencies (8)
Dev Dependencies (10)
Dependents (0)
None

Copyright 2014 - 2016 © taobao.org |