Introduction
------------
You can instruct ntopng to save flow information in Elasticsearch so that you can
create your own reports using an "ENK" stack (Elasticsearch-ntopng-Kibana)


How To Enable It
----------------
In order to enable this feature you need to start ntopng with the -F.

Example: 
$ ntopng -F "es;ntopng;ntopng-%Y.%m.%d;http://localhost:9200/_bulk;"

Syntax:
es;<index type>;<index name>;<ES Bulk API>;{optional username};{optional pass};

Note: the <idx name> accepts the strftime() format.

ntopng will create new indexes automatically so no action is required from your side.
ntopng will NOT create the mappings automatically - a template should be created. 

Example:
$ curl -XPUT localhost:9200/_template/ntopng -d '
    {
      "template" : "ntopng-*",
      "settings" : {
        "index.refresh_interval" : "5s"
      },
      "mappings" : {
        "_default_" : {
           "_all" : {"enabled" : true, "omit_norms" : true},
           "dynamic_templates" : [ {
                 "string_fields" : {
                   "match" : "*",
                   "match_mapping_type" : "string",
                   "mapping" : {
                     "type" : "string", "index" : "analyzed", "omit_norms" : true,
                       "fields" : {
                         "raw" : {"type": "string", "index" : "not_analyzed", "ignore_above" : 256}
                       }
                   }
                 }
           }, {
                 "geo_fields" : {
                   "match" : "*_IP_LOCATION",
                   "mapping": {
                          "type": "geo_point"
                    }
                 }
           }, {
                 "ip_fields" : {
                   "match" : "IPV4_*",
                   "match_mapping_type" : "string",
                   "mapping": {
                          "type": "ip"
                    }
                 }
           } ],
           "properties" : {
             "@version": { "type": "string", "index": "not_analyzed" }
           }
        }
      }
    }'


Using It
--------
Enabling this option will cause ntopng dump flow information in Elasticsearch
and allows the user/integrator to recreate statistics and monitoring logic using
Kibana or other ES Clients in parallel ntopng's own interface. 

Once started, ntopng will push to ES flows that are expired or periodically send 
(every 5 mins) partial flows for long lasting flows.

ES Information is currently not used back by ntopng, integration with historical
views is planned in upcoming versions.


Data Retention
--------------
Flows from ntopng typically use a date-stamped index (ie ntopng-%Y.%m.%d ).
You can use Elasticsearch Curator to delete indexes over the retention period. 

Example:
$ curator --host localhost delete indices --older-than 30 /
  --time-unit days --timestring '%Y.%m.%d'

Indexes can also be manually removed during testing:
$ curl -XDELETE 'http://localhost:9200/ntopng-2015.09.27/'

To remove all Indexes generated by ntopng (according to standard formats):
$ curl -XDELETE 'http://localhost:9200/ntopng-*/'

Future Work
-----------
In the future we plan:

- adding automated mapping and rotation of ES indexes
- adding ElasticSearch support to historical view of ntopng