# NAME App::ElasticSearch::Utilities - Utilities for Monitoring ElasticSearch # VERSION version 4.2 # SYNOPSIS This library contains utilities for unified interfaces in the scripts. This a set of utilities to make monitoring ElasticSearch clusters much simpler. Included are: **SEARCHING**: scripts/es-search.pl - Utility to interact with LogStash style indices from the CLI **MONITORING**: scripts/es-nagios-check.pl - Monitor ES remotely or via NRPE with this script scripts/es-graphite-dynamic.pl - Perform index maintenance on daily indexes scripts/es-status.pl - Command line utility for ES Metrics scripts/es-storage-data.pl - View how shards/data is aligned on your cluster scripts/es-nodes.pl - View node information **MAINTENANCE**: scripts/es-daily-index-maintenance.pl - Perform index maintenance on daily indexes scripts/es-alias-manager.pl - Manage index aliases automatically scripts/es-open.pl - Open any closed indices matching a index parameters **MANAGEMENT**: scripts/es-copy-index.pl - Copy an index from one cluster to another scripts/es-apply-settings.pl - Apply settings to all indexes matching a pattern scripts/es-storage-data.pl - View how shards/data is aligned on your cluster **DEPRECATED**: scripts/es-graphite-static.pl - Send ES Metrics to Graphite or Cacti The App::ElasticSearch::Utilities module simply serves as a wrapper around the scripts for packaging and distribution. # FUNCTIONS ## es\_globals($key) Grab the value of the global value from the es-utils.yaml files. ## es\_pattern Returns a hashref of the pattern filter used to get the indexes { string => '\*', re => '.\*', } ## es\_connect Without options, this connects to the server defined in the args. If passed an array ref, it will use that as the connection definition. ## es\_request(\[$handle\],$command,{ method => 'GET', parameters => { a => 1 } }, {}) Retrieve URL from ElasticSearch, returns a hash reference First hash ref contains options, including: uri_param Query String Parameters index Index name type Index type method Default is GET ## es\_nodes Returns the hash of index meta data. ## es\_indices\_meta Returns the hash of index meta data. ## es\_indices Returns a list of active indexes matching the filter criteria specified on the command line. Can handle indices named: logstash-YYYY.MM.DD dcid-logstash-YYYY.MM.DD logstash-dcid-YYYY.MM.DD logstash-YYYY.MM.DD-dcid Makes use of --datesep to determine where the date is. Options include: - **state** Default is 'open', can be used to find 'closed' indexes as well. - **check\_state** Default is 1, set to 0 to disable state checks. The combination of the default with this option and the default for **state** means only open indices are returned. - **check\_dates** Default is 1, set to 0 to disable checking index age. ## es\_index\_strip\_date( 'index-name' ) Returns the index name with the date removed. ## es\_index\_days\_old( 'index-name' ) Return the number of days old this index is. ## es\_index\_shard\_replicas( 'index-name' ) Returns the number of replicas for a given index. ## es\_index\_valid( 'index-name' ) Checks if the specified index is valid ## es\_index\_fields('index-name') Returns a list of the fields in a given index. ## es\_close\_index('index-name') Closes an index ## es\_open\_index('index-name') Open an index ## es\_delete\_index('index-name') Deletes an index ## es\_optimize\_index('index-name') Optimize an index to a single segment per shard ## es\_index\_segments( 'index-name' ) Exposes GET /$index/\_segments Returns the segment data from the index in hashref: ## es\_segment\_stats($index) Return the number of shards and segments in an index as a hashref ## es\_index\_stats( 'index-name' ) Exposes GET /$index/\_stats Returns a hashref ## es\_settings() Exposes GET /\_settings Returns a hashref ## es\_node\_stats() Exposes GET /\_nodes/stats Returns a hashref ## def('key') Exposes Definitions grabbed by options parsing # ARGS From App::ElasticSearch::Utilities: --local Use localhost as the elasticsearch host --host ElasticSearch host to connect to --port HTTP port for your cluster --noop Any operations other than GET are disabled, can be negated with --no-noop --timeout Timeout to ElasticSearch, default 30 --keep-proxy Do not remove any proxy settings from %ENV --index Index to run commands against --base For daily indexes, reference only those starting with "logstash" (same as --pattern logstash-* or logstash-DATE) --datesep Date separator, default '.' also (--date-separator) --pattern Use a pattern to operate on the indexes --days If using a pattern or base, how many days back to go, default: all # ARGUMENT GLOBALS Some options may be specified in the **/etc/es-utils.yaml** or **$HOME/.es-utils.yaml** file: --- host: esproxy.example.com port: 80 timeout: 10 # INSTALL **This library attempts to provide scripts compatible with version 0.19 through 1.1 of ElasticSearch**. Recommended install with [CPAN Minus](http://cpanmin.us): cpanm App::ElasticSearch::Utilities You can also use CPAN: cpan App::ElasticSearch::Utilities Or if you'd prefer to manually install: export RELEASE= wget --no-check-certificate https://github.com/reyjrar/es-utils/blob/master/releases/App-ElasticSearch-Utilities-$RELEASE.tar.gz?raw=true -O es-utils.tgz tar -zxvf es-utils.tgz cd App-ElasticSearch-Utilities-$RELEASE perl Makefile.PL make make install This will take care of ensuring all the dependencies are satisfied and will install the scripts into the same directory as your Perl executable. ## USAGE The tools are all wrapped in their own documentation, please see: $UTILITY --help $UTILITY --manual For individual options and capabilities ## PATTERNS Patterns are used to match an index to the aliases it should have. A few symbols are expanded into regular expressions. Those patterns are: * expands to match any number of any characters. DATE expands to match YYYY.MM.DD, YYYY-MM-DD, or YYYYMMDD ANY expands to match any number of any characters. # AUTHOR Brad Lhotsky <brad@divisionbyzero.net> # COPYRIGHT AND LICENSE This software is Copyright (c) 2012 by Brad Lhotsky. This is free software, licensed under: The (three-clause) BSD License # CONTRIBUTORS - Alexey Shatlovsky <alexey.shatlovsky@booking.com> - Alexey Surikov <ksurent@gmail.com> - Daniel Ostermeier <daniel.ostermeier@gmail.com> - Jason Rojas <jason.rojas@mgo.com> - Kang-min Liu <gugod@gugod.org> - Samit Badle - Takumi Sakamoto <takumi.saka@gmail.com> # SUPPORT ## Websites The following websites have more information about this module, and may be of help to you. As always, in addition to those websites please use your favorite search engine to discover more resources. - MetaCPAN A modern, open-source CPAN search engine, useful to view POD in HTML format. [http://metacpan.org/release/App-ElasticSearch-Utilities](http://metacpan.org/release/App-ElasticSearch-Utilities) - RT: CPAN's Bug Tracker The RT ( Request Tracker ) website is the default bug/issue tracking system for CPAN. [https://rt.cpan.org/Public/Dist/Display.html?Name=App-ElasticSearch-Utilities](https://rt.cpan.org/Public/Dist/Display.html?Name=App-ElasticSearch-Utilities) ## Source Code This module's source code is available by visiting: [https://github.com/reyjrar/es-utils](https://github.com/reyjrar/es-utils)