We use proprietary and third party´s cookies to improve your experience and our services, identifying your Internet Browsing preferences on our website; develop analytic activities and display advertising based on your preferences. If you keep browsing, you accept its use. You can get more information on our Cookie Policy
Cookies Policy
Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

You will have to use WebHDFS in order to browse your HDFS user space. Detailed referece for WebHDFS can be found at https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/WebHDFS.html.

For instance, by using my HDFS user "frb" I can do:

$ curl -X GET "http://cosmos.lab.fiware.org:14000/webhdfs/v1/user/frb?op=liststatus&user.name=frb" -H "X-Auth-Token: <MY_TOKEN>" | python -m json.tool { "FileStatuses": { "FileStatus": [ { "accessTime": 0, "blockSize": 0, "group": "frb", "length": 0, "modificationTime": 1455191589436, "owner": "frb", "pathSuffix": "-p", "permission": "755", "replication": 0, "type": "DIRECTORY" }, { "accessTime": 0, "blockSize": 0, "group": "frb", "length": 0, "modificationTime": 1456152801883, "owner": "frb", "pathSuffix": "FRB", "permission": "740", "replication": 0, "type": "DIRECTORY" },...

Cygnus stores the data as:


You can check the Cygnus logs as well in order to know if the data has been successfully persisted. Try finding logs like:

time=2016-03-29T09:55:50.779CEST | lvl=INFO | trans=1459238120-153-0000000000 | svc=default | subsvc=/ | function=persistAggregation | comp=Cygnus | msg=com.telefonica.iot.cygnus.sinks.OrionHDFSSink[954] : [hdfs-sink] Persisting data at OrionHDFSSink. HDFS file (myservice/myservicepath/entityId_entityType/entityId_entityType.txt), Data ({"recvTime":"2016-03-29T07:55:24.441Z","fiwareServicePath":"myservicepath","entityId":"xxx","entityType":"xxx", "xxx":"111", "xxx_md":[]})