We use proprietary and third party´s cookies to improve your experience and our services, identifying your Internet Browsing preferences on our website; develop analytic activities and display advertising based on your preferences. If you keep browsing, you accept its use. You can get more information on our Cookie Policy
Cookies Policy
Ask Your Question
0

Unable to run Cygnus with MySQL agent

asked 2018-05-03 15:32:23 +0200

babbarkrishan gravatar image

Hi All,

I am trying to setup and understand Cygnus. But I am facing issue during installation.

I followed below given steps.

  1. Install Cygnus using Docker (docker run -d -p 5050:5050 -p 8081:8081 fiware/cygnus-common)
  2. Executed version command (curl http://172.17.0.2:8081/v1/version) which gave following response {"success":"true","version":"1.8.0_SNAPSHOT.39b2aa4789c61fa92fe6edc905410f1ddeb33490"}
  3. Login into Cygnus container using command docker exec -it /bin/bash
  4. Created new file named “agent_mysql.conf” in “/opt/apache-flume/conf/” folder.


Configuration details given below

cygnus-ngsi.sources = http-source
cygnus-ngsi.sinks = mysql-sink
cygnus-ngsi.channels = mysql-channel

cygnus-ngsi.sources.http-source.channels = mysql-channel
cygnus-ngsi.sources.http-source.type = org.apache.flume.source.http.HTTPSource
cygnus-ngsi.sources.http-source.port = 5050
cygnus-ngsi.sources.http-source.handler = com.telefonica.iot.cygnus.handlers.NGSIRestHandler
cygnus-ngsi.sources.http-source.handler.notificationtarget = /notify
cygnus-ngsi.sources.http-source.handler.default
service = defserv
cygnus-ngsi.sources.http-source.handler.default
servicepath = defservpath
cygnus-ngsi.sources.http-source.handler.eventsttl = 2
cygnus-ngsi.sources.http-source.interceptors = ts gi
cygnus-ngsi.sources.http-source.interceptors.ts.type = timestamp
cygnus-ngsi.sources.http-source.interceptors.gi.type = com.telefonica.iot.cygnus.interceptors.NGSIGroupingInterceptor$Builder
cygnus-ngsi.sources.http-source.interceptors.gi.grouping
rulesconffile = /Applications/apache-flume-1.4.0-bin/conf/grouping_rules.conf

cygnus-ngsi.channels.mysql-channel.type = memory

cygnus-ngsi.channels.mysql-channel.capacity = 1000

cygnus-ngsi.channels.mysql-channel.transactionCapacity = 100


cygnus-ngsi.sinks.mysql-sink.channel = mysql-channel
cygnus-ngsi.sinks.mysql-sink.type = com.telefonica.iot.cygnus.sinks.NGSIMySQLSink
cygnus-ngsi.sinks.mysql-sink.mysqlhost = localhost
cygnus-ngsi.sinks.mysql-sink.mysql
port = 3306
cygnus-ngsi.sinks.mysql-sink.mysqlusername = root
cygnus-ngsi.sinks.mysql-sink.mysql
password = <mypassword>
cygnus-ngsi.sinks.mysql-sink.attr_persistence = row


  1. Changed "cygnus-entrypoint.sh" file in / (root) folder and added following command by removing existing one.
    ${FLUMEHOME}/bin/cygnus-flume-ng agent --conf ${CYGNUSCONFPATH} -f ${CYGNUSCONFPATH}/agentmysql.conf -n cygnus-ngsi -p ${CYGNUSAPIPORT} -Dflume.root.logger=${CYGNUSLOGLEVEL},${CYGNUSLOGAPPENDER} -Dfile.encoding=UTF-8
  2. Exited Docker container and came back to Ubuntu.
  3. Stop and restart Docker container.
  4. And I am getting following errors in logs


Please check and let me know what am I doing wrong? Appreciate your help.

LOGS



n$AgentConfiguration[1016] : Processing:mysql-sink time=2018-04-30T14:24:00.807Z | lvl=INFO | corr=N/A | trans=N/A | srv=N/A | subsrv=N/A | comp=cygnus-ngsi | op=validateConfiguration | msg=org.apache.flume.conf.FlumeConfiguration[140] : Post-validation flume configuration contains configuration for agents: [cygnus-ngsi] time=2018-04-30T14:24:00.808Z | lvl=INFO | corr=N/A | trans=N/A | srv=N/A | subsrv=N/A | comp=cygnus-ngsi | op=loadChannels | msg=org.apache.flume.node.AbstractConfigurationProvider[150] : Creating channels time=2018-04-30T14:24:00.816Z | lvl=INFO | corr=N/A | trans=N/A | srv=N/A | subsrv=N/A | comp=cygnus-ngsi | op=create | msg=org.apache.flume.channel.DefaultChannelFactory[40] : Creating instance of channel mysql-channel type memory time=2018-04-30T14:24:00.825Z | lvl=INFO | corr=N/A | trans=N/A | srv=N/A | subsrv=N/A | comp=cygnus-ngsi | op=loadChannels | msg=org.apache.flume.node.AbstractConfigurationProvider[205] : Created channel mysql-channel time=2018-04-30T14:24:00.832Z | lvl=INFO | corr=N/A | trans=N/A ...

(more)

edit retag flag offensive close merge delete

1 answer

Sort by » oldest newest most voted
0

answered 2018-05-14 12:02:10 +0200

babbarkrishan gravatar image

Added my answer at https://stackoverflow.com/questions/5...

Finally I am able to run Cygnus with MySQL agent. I am using Ubuntu. (Linux ubuntucustomfiware 4.4.0-119-generic #143-Ubuntu SMP Mon Apr 2 16:08:24 UTC 2018 x8664 x8664 x86_64 GNU/Linux)

I followed below steps.

Used MySQL installed in main Ubuntu instance instead of Docker container. Modified /etc/mysql/mysql.conf.d/mysqld.cnf and changed

from

bind-address = 127.0.0.1

to

bind-address = *

Login into DB and grant all privileges to root user, so that it can connect from any host.

mysql -u root -p

GRANT ALL PRIVILEGES ON . TO 'root'@'%' IDENTIFIED BY 'MyPassword';

FLUSH PRIVILEGES;

exit;

Restart MySQL server

service mysql restart

Run Cygnus-ngsi

docker run -d --name cygnus -p 8081:8081 -p 5050:5050 -e CYGNUSMYSQLHOST=PublicIPOfMySQLServer -e CYGNUSMYSQLPORT=3306 -e CYGNUSMYSQLUSER=root -e CYGNUSMYSQLPASS=MyPassword -e CYGNUSLOGLEVEL='DEBUG' fiware/cygnus-ngsi

Modified Agent file and keep only mysql-sink. After below changes, stop/start cygnus docker container.

docker exec -it cygnus /bin/bash

vi /opt/apache-flume/conf/agent.conf

cygnus-ngsi.sinks = mysql-sink

cygnus-ngsi.channels = mysql-channel

exit;

docker stop cygnus

docker start cygnus

Now publish MQTT data to modify my entity and it inserted 4 rows (one row for each attribute) into MySQL DB

mosquitto_pub -h PublicIPOfMySQLServer -u UserName -P Password -t /swm-reader-service1/reader-device-id1/attrs -m '{"tn": "9888", "pn": "878787", "ri": "888888", "tdt":"Monday, May 10, 2018 03:16 AM"}'

Thanks for all your support.

Regards,

Krishan

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2018-05-03 15:32:23 +0200

Seen: 709 times

Last updated: May 14 '18