rpm/telegraf
SHA256
1
0
Fork 0
forked from pool/telegraf
Code Pull Requests Activity

Accepting request 791105 from home:jfkw:branches:devel:languages:go

Browse Source 
- Update to telegraf-1.13.4
- Remove carried patch for Generic SQL output plugin

OBS-URL: https://build.opensuse.org/request/show/791105
OBS-URL: https://build.opensuse.org/package/show/devel:languages:go/telegraf?expand=0&rev=23
This commit is contained in:
Jeff Kowalczyk 2020-04-03 07:04:24 +00:00 committed by Git OBS Bridge
parent 9a9161a9b6
commit fc34d7e003
7 changed files with 18 additions and 486 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

24
0001-plugins-outputs-all-all.go-Generic-SQL-output-plugin.patch
View File

@ -1,24 +0,0 @@
From 509aa47a92a996bb374a5dea04d310ddfd59a042 Mon Sep 17 00:00:00 2001
From: Johannes Kastl <kastl@b1-systems.de>
Date: Wed, 26 Feb 2020 16:37:21 +0100
Subject: [PATCH] plugins/outputs/all/all.go: Generic SQL output plugin for
Telegraf (patch taken from openSUSE RPM package of telegraf 1.12.6)
Signed-off-by: Johannes Kastl <kastl@b1-systems.de>
---
plugins/outputs/all/all.go | 1 +
1 file changed, 1 insertion(+)
diff --git a/plugins/outputs/all/all.go b/plugins/outputs/all/all.go
index e40230993b51acf3b0ff9cfd06a8d36d7f4938d4..7f5139cdd3f99c1a0892f113d663b59faba562dc 100644
--- a/plugins/outputs/all/all.go
+++ b/plugins/outputs/all/all.go
@@ -34,4 +34,5 @@ import (
_ "github.com/influxdata/telegraf/plugins/outputs/stackdriver"
_ "github.com/influxdata/telegraf/plugins/outputs/syslog"
_ "github.com/influxdata/telegraf/plugins/outputs/wavefront"
+ _ "github.com/influxdata/telegraf/plugins/outputs/sql"
)
--
2.25.1

94
0002-plugins-outputs-sql-README.md-Generic-SQL-output-plu.patch
View File

@ -1,94 +0,0 @@
From 0b3ff433caa76cd644819cab50301f9a2a5b01a1 Mon Sep 17 00:00:00 2001
From: Johannes Kastl <kastl@b1-systems.de>
Date: Wed, 26 Feb 2020 16:38:01 +0100
Subject: [PATCH] plugins/outputs/sql/README.md: Generic SQL output plugin for
Telegraf (patch taken from openSUSE RPM package of telegraf 1.12.6)
Signed-off-by: Johannes Kastl <kastl@b1-systems.de>
---
plugins/outputs/sql/README.md | 73 +++++++++++++++++++++++++++++++++++
1 file changed, 73 insertions(+)
create mode 100644 plugins/outputs/sql/README.md
diff --git a/plugins/outputs/sql/README.md b/plugins/outputs/sql/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..2d29af83fdcc5afc69d2d38002e02ae852223f8f
--- /dev/null
+++ b/plugins/outputs/sql/README.md
@@ -0,0 +1,73 @@
+# SQL plugin
+
+The plugin inserts values to SQL various database.
+Supported/integrated drivers are mssql (SQLServer), mysql (MySQL), postgres (Postgres)
+Activable drivers (read below) are all golang SQL compliant drivers (see https://github.com/golang/go/wiki/SQLDrivers): for instance oci8 for Oracle or sqlite3 (SQLite)
+
+## Getting started :
+First you need to grant insert (if auto create table create) privileges to the database user you use for the connection
+
+## Configuration:
+
+```
+# Send metrics to SQL-Database (Example configuration for MySQL/MariaDB)
+[[outputs.sql]]
+ ## Database Driver, required.
+ ## Valid options: mssql (SQLServer), mysql (MySQL), postgres (Postgres), sqlite3 (SQLite), [oci8 ora.v4 (Oracle)]
+ driver = "mysql"
+
+ ## specify address via a url matching:
+ ## postgres://[pqgotest[:password]]@localhost[/dbname]\
+ ## ?sslmode=[disable|verify-ca|verify-full]
+ ## or a simple string:
+ ## host=localhost user=pqotest password=... sslmode=... dbname=app_production
+ ##
+ ## All connection parameters are optional.
+ ##
+ ## Without the dbname parameter, the driver will default to a database
+ ## with the same name as the user. This dbname is just for instantiating a
+ ## connection with the server and doesn't restrict the databases we are trying
+ ## to grab metrics for.
+ ##
+ address = "username:password@tcp(server:port)/table"
+
+ ## Available Variables:
+ ## {TABLE} - tablename as identifier
+ ## {TABLELITERAL} - tablename as string literal
+ ## {COLUMNS} - column definitions
+ ## {KEY_COLUMNS} - comma-separated list of key columns (time + tags)
+ ##
+
+ ## Check with this is table exists
+ ##
+ ## Template for MySQL is "SELECT 1 FROM {TABLE} LIMIT 1"
+ ##
+ table_exists_template = "SELECT 1 FROM {TABLE} LIMIT 1"
+
+ ## Template to use for generating tables
+
+ ## Default template
+ ##
+ # table_template = "CREATE TABLE {TABLE}({COLUMNS})"
+
+ ## Convert Telegraf datatypes to these types
+ [[outputs.sql.convert]]
+ integer = "INT"
+ real = "DOUBLE"
+ text = "TEXT"
+ timestamp = "TIMESTAMP"
+ defaultvalue = "TEXT"
+ unsigned = "UNSIGNED"
+```
+sql_script is read only once, if you change the script you need to reload telegraf
+
+## Field names
+If database table is not pre-created tries driver to create database. There can be errors as
+SQL has strict scheming.
+
+## Tested Databases
+Actually I run the plugin using MySQL
+
+## TODO
+1) Test with other databases
+2) More sane testing
--
2.25.1

277
0003-plugins-outputs-sql-sql.go-Generic-SQL-output-plugin.patch
View File

@ -1,277 +0,0 @@
From bca63ce35687103b0f52898936d07a1c294673ad Mon Sep 17 00:00:00 2001
From: Johannes Kastl <kastl@b1-systems.de>
Date: Wed, 26 Feb 2020 16:38:59 +0100
Subject: [PATCH] plugins/outputs/sql/sql.go: Generic SQL output plugin for
Telegraf (patch taken from openSUSE RPM package of telegraf 1.12.6)
Signed-off-by: Johannes Kastl <kastl@b1-systems.de>
---
plugins/outputs/sql/sql.go | 256 +++++++++++++++++++++++++++++++++++++
1 file changed, 256 insertions(+)
create mode 100644 plugins/outputs/sql/sql.go
diff --git a/plugins/outputs/sql/sql.go b/plugins/outputs/sql/sql.go
new file mode 100644
index 0000000000000000000000000000000000000000..7823f680008f42f1346d2fea4215ffc664fcfa8c
--- /dev/null
+++ b/plugins/outputs/sql/sql.go
@@ -0,0 +1,256 @@
+package sql
+
+import (
+ "database/sql"
+ "fmt"
+ "log"
+ "strings"
+
+ _ "github.com/go-sql-driver/mysql"
+ _ "github.com/jackc/pgx"
+ // These SQL drivers can be enabled if
+ // they are added to depencies
+ // _ "github.com/lib/pq"
+ // _ "github.com/mattn/go-sqlite3"
+ // _ "github.com/zensqlmonitor/go-mssqldb"
+
+ "github.com/influxdata/telegraf"
+ "github.com/influxdata/telegraf/plugins/outputs"
+)
+
+type ConvertStruct struct {
+ Integer string
+ Real string
+ Text string
+ Timestamp string
+ Defaultvalue string
+ Unsigned string
+}
+
+type Sql struct {
+ db *sql.DB
+ Driver string
+ Address string
+ TableTemplate string
+ TableExistsTemplate string
+ TagTableSuffix string
+ Tables map[string]bool
+ Convert []ConvertStruct
+}
+
+func (p *Sql) Connect() error {
+ db, err := sql.Open(p.Driver, p.Address)
+ if err != nil {
+ return err
+ }
+ p.db = db
+ p.Tables = make(map[string]bool)
+
+ return nil
+}
+
+func (p *Sql) Close() error {
+ return p.db.Close()
+}
+
+func contains(haystack []string, needle string) bool {
+ for _, key := range haystack {
+ if key == needle {
+ return true
+ }
+ }
+ return false
+}
+
+func quoteIdent(name string) string {
+ return name
+}
+
+func quoteLiteral(name string) string {
+ return "'" + strings.Replace(name, "'", "''", -1) + "'"
+}
+
+func (p *Sql) deriveDatatype(value interface{}) string {
+ var datatype string
+
+ switch value.(type) {
+ case int64:
+ datatype = p.Convert[0].Integer
+ case uint64:
+ datatype = fmt.Sprintf("%s %s", p.Convert[0].Integer, p.Convert[0].Unsigned)
+ case float64:
+ datatype = p.Convert[0].Real
+ case string:
+ datatype = p.Convert[0].Text
+ default:
+ datatype = p.Convert[0].Defaultvalue
+ log.Printf("E! Unknown datatype: '%T' %v", value, value)
+ }
+ return datatype
+}
+
+var sampleConfig = `
+# Send metrics to SQL-Database (Example configuration for MySQL/MariaDB)
+[[outputs.sql]]
+ ## Database Driver, required.
+ ## Valid options: mssql (SQLServer), mysql (MySQL), postgres (Postgres), sqlite3 (SQLite), [oci8 ora.v4 (Oracle)]
+ driver = "mysql"
+
+ ## specify address via a url matching:
+ ## postgres://[pqgotest[:password]]@localhost[/dbname]\
+ ## ?sslmode=[disable|verify-ca|verify-full]
+ ## or a simple string:
+ ## host=localhost user=pqotest password=... sslmode=... dbname=app_production
+ ##
+ ## All connection parameters are optional.
+ ##
+ ## Without the dbname parameter, the driver will default to a database
+ ## with the same name as the user. This dbname is just for instantiating a
+ ## connection with the server and doesn't restrict the databases we are trying
+ ## to grab metrics for.
+ ##
+ address = "username:password@tcp(server:port)/table"
+
+ ## Available Variables:
+ ## {TABLE} - tablename as identifier
+ ## {TABLELITERAL} - tablename as string literal
+ ## {COLUMNS} - column definitions
+ ## {KEY_COLUMNS} - comma-separated list of key columns (time + tags)
+ ##
+
+ ## Check with this is table exists
+ ##
+ ## Template for MySQL is "SELECT 1 FROM {TABLE} LIMIT 1"
+ ##
+ table_exists_template = "SELECT 1 FROM {TABLE} LIMIT 1"
+
+ ## Template to use for generating tables
+
+ ## Default template
+ ##
+ # table_template = "CREATE TABLE {TABLE}({COLUMNS})"
+
+ ## Convert Telegraf datatypes to these types
+ [[outputs.sql.convert]]
+ integer = "INT"
+ real = "DOUBLE"
+ text = "TEXT"
+ timestamp = "TIMESTAMP"
+ defaultvalue = "TEXT"
+ unsigned = "UNSIGNED"
+`
+
+func (p *Sql) SampleConfig() string { return sampleConfig }
+func (p *Sql) Description() string { return "Send metrics to SQL Database" }
+
+func (p *Sql) generateCreateTable(metric telegraf.Metric) string {
+ var columns []string
+ var pk []string
+ var sql []string
+
+ pk = append(pk, quoteIdent("timestamp"))
+ columns = append(columns, fmt.Sprintf("timestamp %s", p.Convert[0].Timestamp))
+
+ // handle tags if necessary
+ if len(metric.Tags()) > 0 {
+ // tags in measurement table
+ for column := range metric.Tags() {
+ pk = append(pk, quoteIdent(column))
+ columns = append(columns, fmt.Sprintf("%s %s", quoteIdent(column), p.Convert[0].Text))
+ }
+ }
+
+ var datatype string
+ for column, v := range metric.Fields() {
+ datatype = p.deriveDatatype(v)
+ columns = append(columns, fmt.Sprintf("%s %s", quoteIdent(column), datatype))
+ }
+
+ query := strings.Replace(p.TableTemplate, "{TABLE}", quoteIdent(metric.Name()), -1)
+ query = strings.Replace(query, "{TABLELITERAL}", quoteLiteral(metric.Name()), -1)
+ query = strings.Replace(query, "{COLUMNS}", strings.Join(columns, ","), -1)
+ query = strings.Replace(query, "{KEY_COLUMNS}", strings.Join(pk, ","), -1)
+
+ sql = append(sql, query)
+ return strings.Join(sql, ";")
+}
+
+func (p *Sql) generateInsert(tablename string, columns []string) string {
+
+ var placeholder, quoted []string
+ for _, column := range columns {
+ placeholder = append(placeholder, fmt.Sprintf("?"))
+ quoted = append(quoted, quoteIdent(column))
+ }
+
+ sql := fmt.Sprintf("INSERT INTO %s(%s) VALUES(%s)", quoteIdent(tablename), strings.Join(quoted, ","), strings.Join(placeholder, ","))
+ return sql
+}
+
+func (p *Sql) tableExists(tableName string) bool {
+ stmt := strings.Replace(p.TableExistsTemplate, "{TABLE}", quoteIdent(tableName), -1)
+
+ _, err := p.db.Exec(stmt)
+ if err != nil {
+ return false
+ }
+ return true
+}
+
+func (p *Sql) Write(metrics []telegraf.Metric) error {
+ for _, metric := range metrics {
+ tablename := metric.Name()
+
+ // create table if needed
+ if p.Tables[tablename] == false && p.tableExists(tablename) == false {
+ createStmt := p.generateCreateTable(metric)
+ _, err := p.db.Exec(createStmt)
+ if err != nil {
+ return err
+ }
+ p.Tables[tablename] = true
+ }
+
+ var columns []string
+ var values []interface{}
+
+ // We assume that SQL is making auto timestamp
+ //columns = append(columns, "timestamp")
+ //values = append(values, metric.Time())
+
+ if len(metric.Tags()) > 0 {
+ // tags in measurement table
+ for column, value := range metric.Tags() {
+ columns = append(columns, column)
+ values = append(values, value)
+ }
+ }
+
+ for column, value := range metric.Fields() {
+ columns = append(columns, column)
+ values = append(values, value)
+ }
+
+ sql := p.generateInsert(tablename, columns)
+ _, err := p.db.Exec(sql, values...)
+
+ if err != nil {
+ // check if insert error was caused by column mismatch
+ log.Printf("E! Error during insert: %v", err)
+ return err
+ }
+ }
+ return nil
+}
+
+func init() {
+ outputs.Add("sql", func() telegraf.Output { return newSql() })
+}
+
+func newSql() *Sql {
+ return &Sql{
+ TableTemplate: "CREATE TABLE {TABLE}({COLUMNS})",
+ TableExistsTemplate: "SELECT 1 FROM {TABLE} LIMIT 1",
+ TagTableSuffix: "_tag",
+ }
+}
--
2.25.1

50
0004-plugins-outputs-sql-sql_test.go-Generic-SQL-output-p.patch
View File

@ -1,50 +0,0 @@
From a79b89f1037f705d606374b63f69e82406d42a4c Mon Sep 17 00:00:00 2001
From: Johannes Kastl <kastl@b1-systems.de>
Date: Wed, 26 Feb 2020 16:40:19 +0100
Subject: [PATCH] plugins/outputs/sql/sql_test.go: Generic SQL output plugin
for Telegraf (patch taken from openSUSE RPM package of telegraf 1.12.6)
Signed-off-by: Johannes Kastl <kastl@b1-systems.de>
---
plugins/outputs/sql/sql_test.go | 29 +++++++++++++++++++++++++++++
1 file changed, 29 insertions(+)
create mode 100644 plugins/outputs/sql/sql_test.go
diff --git a/plugins/outputs/sql/sql_test.go b/plugins/outputs/sql/sql_test.go
new file mode 100644
index 0000000000000000000000000000000000000000..80d3b7104f9c42f46d75f2335a73f237282275e8
--- /dev/null
+++ b/plugins/outputs/sql/sql_test.go
@@ -0,0 +1,29 @@
+package sql
+
+import (
+ "testing"
+ // "time"
+ // "github.com/influxdata/telegraf"
+ // "github.com/influxdata/telegraf/metric"
+ // "github.com/stretchr/testify/assert"
+)
+
+func TestSqlQuote(t *testing.T) {
+ if testing.Short() {
+ t.Skip("Skipping integration test in short mode")
+ }
+
+}
+
+func TestSqlCreateStatement(t *testing.T) {
+ if testing.Short() {
+ t.Skip("Skipping integration test in short mode")
+ }
+
+}
+
+func TestSqlInsertStatement(t *testing.T) {
+ if testing.Short() {
+ t.Skip("Skipping integration test in short mode")
+ }
+}
--
2.25.1

26
0005-README.md-Generic-SQL-output-plugin-for-Telegraf-pat.patch
View File

@ -1,26 +0,0 @@
From bc09eb99d169e1122bc7fe0eb6f85140c736a243 Mon Sep 17 00:00:00 2001
From: Johannes Kastl <kastl@b1-systems.de>
Date: Wed, 26 Feb 2020 16:57:32 +0100
Subject: [PATCH] README.md: Generic SQL output plugin for Telegraf (patch
taken from openSUSE RPM package of telegraf 1.12.6)
Signed-off-by: Johannes Kastl <kastl@b1-systems.de>
---
README.md | 1 +
1 file changed, 1 insertion(+)
diff --git a/README.md b/README.md
index b34451df2a568c2f2eeede165b2a48d19b8fe7d9..54aa6988ebc987a6566a24200f8830ec48748b2a 100644
--- a/README.md
+++ b/README.md
@@ -285,6 +285,7 @@ For documentation on the latest development code see the [documentation index][d
* [snmp_trap](./plugins/inputs/snmp_trap)
* [socket_listener](./plugins/inputs/socket_listener)
* [solr](./plugins/inputs/solr)
+* [sql](./plugins/outputs/sql) (sql generic output)
* [sql server](./plugins/inputs/sqlserver) (microsoft)
* [stackdriver](./plugins/inputs/stackdriver)
* [statsd](./plugins/inputs/statsd)
--
2.25.1

18
telegraf.changes
View File

@ -1,3 +1,21 @@
-------------------------------------------------------------------
Fri Apr 3 06:36:08 UTC 2020 - Jeff Kowalczyk <jkowalczyk@suse.com>
- Remove carried patch for Generic SQL output plugin. The upstream PR
https://github.com/influxdata/telegraf/pull/4205 that is the source of this
patch hasn't accepted it since 2018, with no recent activity. Dropping the
patch as there are no known users.
* drop 0001-Generic-SQL-output-plugin-for-Telegraf.patch unified original
* drop 0001-plugins-outputs-all-all.go-Generic-SQL-output-plugin.patch split
* drop 0002-plugins-outputs-sql-README.md-Generic-SQL-output-plu.patch split
* drop 0003-plugins-outputs-sql-sql.go-Generic-SQL-output-plugin.patch split
* drop 0004-plugins-outputs-sql-sql_test.go-Generic-SQL-output-p.patch split
* drop 0005-README.md-Generic-SQL-output-plugin-for-Telegraf-pat.patch split
* packaging: telegraf-1.13.4 still uses the dep dependency tool, now
documented as a comment in the .spec. An upcoming telegraf release will
complete the transition to go modules, at which point packaging will be
updated accordingly.
-------------------------------------------------------------------
Wed Mar 4 19:13:12 UTC 2020 - Johannes Kastl <kastl@b1-systems.de>

15
telegraf.spec
View File

@ -33,16 +33,6 @@ BuildRequires: golang-packaging
BuildRequires: systemd-rpm-macros
%{?systemd_ordering}
## Features ##
Patch1: 0001-plugins-outputs-all-all.go-Generic-SQL-output-plugin.patch
Patch2: 0002-plugins-outputs-sql-README.md-Generic-SQL-output-plu.patch
Patch3: 0003-plugins-outputs-sql-sql.go-Generic-SQL-output-plugin.patch
Patch4: 0004-plugins-outputs-sql-sql_test.go-Generic-SQL-output-p.patch
Patch5: 0005-README.md-Generic-SQL-output-plugin-for-Telegraf-pat.patch
## /Features ##
%define _influxdata_dir %{_builddir}/src/github.com/influxdata
%define _telegraf_dir %{_influxdata_dir}/%{name}
%define _config_dir %{_sysconfdir}/%{name}
@ -59,11 +49,6 @@ rm -rf %{name}
ln -sf %{name}-%{version} %{name}
cd %{name}
tar -xJf %{SOURCE1}
%patch1 -p1
%patch2 -p1
%patch3 -p1
%patch4 -p1
%patch5 -p1
%build
%ifnarch ppc64 ppc64le