neocrawl

DESCRIPTION

NEOCRAWL is built as a service/microservice that mediates any listing, finding or filtering request between an api and a neo4j database. The communication is done over rest api.
This service allows you to find/list any node from within your neo4j database using related nodes properties (from any relationship distance)
It was build as light as possible and it's only purpose is to generate appropriate cypher queries based on given filters and not intense processing

Issues, Pull requests and Enhancement requests are very welcomed and encouraged ! :D

REQUIREMENTS

1. All neo4j database clients must have version of at least 2.2.0 for the setup part

The search/listing works with lower versions but it's not recommended

It is also recommended to use the latest stable neo4j version

2. Node.js version of at least v4.2.1

In this case it is also recommended to use the latest stable node.js version

3. A unix/linux based environment for the service deployment

DEPLOYMENT

As a node module

Using npm:

npm i --save neocrawl

In Node.js:

var neocrawl = require('neocrawl');

As a microservice

1. Clone the repo

git clone https://github.com/scstan/neocrawl.git

or

git clone git@github.com:scstan/neocrawl.git

2. Install all dependencies

npm install

3. Edit the config.json with appropriate host and port to suit your needs

4. Start the service by starting either of app.js or clusters.js:

node app.js / node cluster.js

USAGE

As a node module

The enlisted enpoints below are transformed into promised methods

ex:
neocrawl
  .setupdb(parameters) //       <== the promise's parameters are exactly the requests body as stated below
  .then(result => do_stuff())
  .catch(err   => log_err())

Methods:

• neocrawl.setupdb(params)
• neocrawl.getgraph(params)
• neocrawl.search(params)

As a microservice

This service provides 3 rest api endpoints.

1. SetupDB

This will generate a .json, in the graphs directory, with all the "models" in your database
Every usage of this endpoint updates the .json with your latest database mapping
It is recommended to use this endpoint everytime you deploy you application
Please note that this may take a bit depending on your database size

POST {{base_url}}/api/setupdb

{
    "dbAlias": "localhost",        // <= this will be the base_name for you .json [MANDATORY]
    "dbUrl": "localhost:7474",     // <= target neo4j database base_url or ip:port [MANDATORY]
    "update": false                // <= if you already setted up a db with the given alias neocrawl will only allow you to update it by sending this
                                        parameter as true [OPTIONAL]
}

2. Get Graph

This will retrieve the generated map previously created using setupdb endpoint

POST {{base_url}}/api/getgraph

{
    "dbAlias": "localhost",        // <= this will be used to locate your mapped database [MANDATORY]
}

Response example [the key in this returned map is the actual label of the node]

{
  ...
  "User": {
    "relatedNodes": [
      "Role",
      "_Role"
    ],
    "properties": [
      ...
      "password",
      "id",
      "email",
      ...
    ]
  },
  ...
}

3. Search

This is the main endpoint which retrieves results base on you request

POST {{base_url}}/api/search

{
    "dbAlias": "localhost",            // <= this will be used to locate your mapped database [MANDATORY]
    "node": "User",                    // <= requested node label [MANDATORY]
    "dbUrl": "http://localhost:7474/", // <= target db [MANDATORY]
    "offset": 1,                       // <= first page aka skip 0 [MANDATORY]
    "limit": 10,                       // <= self explanatory [MANDATORY]
    "orderBy": "idRestaurant",         // <= [OPTIONAL]
    "direction": "asc",                // <= [OPTIONAL]
    "debug": true,                     // <= prints out the generated neo4j cypher query [OPTIONAL]
    "filters": {                       // <= filters list [MANDATORY but can be left empty]
        "User.email": {
            "like": "@gmail"
        },
        "User.lastName": {
            "like":"smith"
        },
        "User.customImage":{
            "has": false
        },
        "Article.date":{between: [12323433545, 1232399999]}
    },
    "return":"{user:User, roles:collect(distinct(Role))}"
              //  <= custom return must be a stringified representation. [OPTIONAL].
}             // if custom return is not provided the search will return a list of ids
              // based on the requested node type

FILTERS

The filter key is composed in the following manner LABEL.PROPERTY_NAME

Filters can be provided either by direct assignment

"User.email": "eric@gmail.com"    // <= this defaults to the eq operator

Or by picking an operator from the bellow list:

ex: "User.lastName": {
            "like":"smith"
        }

'eq': =
'lt': <
'le': <=
'gt': >
'ge': >=
'ne': <>
'in': checks that the given property [array] contains the given value
'out': checks that the given property [array] does not contain the given value
'has': checks if the given property exists on the node [boolean]
'containsAny': checks that ANY of the elements from the value [array] is found in the given property [array]
'containsAll': checks that ALL of the elements from the value [array] is found in the given property [array]
'excludesAny': checks that ANY of the elements from the value [array] are not found in the given property [array]
'excludesAll': checks that ALL of the elements from the value [array] are not found in the given property [array]
'like': checks that property fully or partially contains the given value [number or string | case insensitive]
'regex': checks the given regex against the given property
'between': checks that the given property is ge than the first element of the given value and le than the second element of the given value
'isRelated': checks that the given property is related with the given value (must be real label)   eg. usage: {Cake.node:{isRelated: "Person"}}
'notRelated': checks that the given property is not related to the given value (must be real label) eg. usage: {Cake.node:{notRelated: "Person"}}

Issues, Pull requests and Enhancement requests are very welcomed and encouraged ! :D

The End