6 Commits

Author SHA1 Message Date
Rob Best
3eab6a10fd kubernetes: use namespace selector where possible
We can use a namespace selector to only list secrets in the target namespace,
unless its a glob pattern.
2024-04-30 06:39:10 +01:00
Rob Best
1b8a0c3b93 Use custom User-Agent header (#178) 2024-04-28 18:54:55 +01:00
Rob Best
dd2a9a2e71 Add test for TRUSTED CERTIFICATE block (#177)
Not the best test in the world but at least it verifies that we read
this block into a certificate.
2024-04-28 18:16:04 +01:00
MisterVVP
1ec0cd6dc7 fix: support parsing of openssl specific cert formats (#142) 2024-04-28 17:44:52 +01:00
jaroug
515b990f52 Add http_file prober (#144)
* feat: add remote_file probe

* fix: use tls module config

* chore: write http/https tests for probing remote file

* chore: get rid of useless lines

* fix: get rid of useless file download, check body directly

* fix: use checkCertificateMetrics to actually check values

* Rename remote_file to http_file

You can fetch remote content with a lot of different protocols, so I
think it's worth being specific here.

As part of this change I've fixed up some of the logic in the code. I've
also created a separate `http_file` block in the module config.

* Actually include renamed files

---------

Co-authored-by: Anthony LE BERRE <aleberre@veepee.com>
Co-authored-by: Rob Best <rob.best@jetstack.io>
2024-04-28 16:48:09 +01:00
Rob Best
4cb38cb268 Stop running tests twice on PR (#176) 2024-04-28 16:45:53 +01:00
12 changed files with 366 additions and 10 deletions

View File

@@ -1,6 +1,12 @@
name: test-and-snapshot
on: [push, pull_request]
on:
push:
branches:
- master
pull_request:
branches:
- master
jobs:
test:

View File

@@ -4,6 +4,7 @@ Exports metrics for certificates collected from various sources:
- [TCP probes](#tcp)
- [HTTPS probes](#https)
- [PEM files](#file)
- [Remote PEM files](#http_file)
- [Kubernetes secrets](#kubernetes)
- [Kubeconfig files](#kubeconfig)
@@ -130,7 +131,7 @@ scrape_configs:
```
This will use proxy servers discovered by the environment variables `HTTP_PROXY`,
`HTTPS_PROXY` and `ALL_PROXY`. Or, you can set the `proxy_url` option in the module
`HTTPS_PROXY` and `ALL_PROXY`. Or, you can set the `https.proxy_url` option in the module
configuration.
The latter takes precedence.
@@ -175,6 +176,44 @@ scrape_configs:
replacement: ${1}:9219
```
### HTTP File
The `http_file` prober exports `ssl_cert_not_after` and
`ssl_cert_not_before` for PEM encoded certificates found at the
specified URL.
```
curl "localhost:9219/probe?module=http_file&target=https://www.paypalobjects.com/marketing/web/logos/paypal_com.pem"
```
Here's a sample Prometheus configuration:
```yml
scrape_configs:
- job_name: 'ssl-http-files'
metrics_path: /probe
params:
module: ["http_file"]
static_configs:
- targets:
- 'https://www.paypalobjects.com/marketing/web/logos/paypal_com.pem'
- 'https://d3frv9g52qce38.cloudfront.net/amazondefault/amazon_web_services_inc_2024.pem'
relabel_configs:
- source_labels: [__address__]
target_label: __param_target
- source_labels: [__param_target]
target_label: instance
- target_label: __address__
replacement: 127.0.0.1:9219
```
For proxying to the target resource, this prober will use proxy servers
discovered in the environment variables `HTTP_PROXY`, `HTTPS_PROXY` and
`ALL_PROXY`. Or, you can set the `http_file.proxy_url` option in the module
configuration.
The latter takes precedence.
### Kubernetes
The `kubernetes` prober exports `ssl_kubernetes_cert_not_after` and
@@ -293,6 +332,7 @@ target: <string>
[ https: <https_probe> ]
[ tcp: <tcp_probe> ]
[ kubernetes: <kubernetes_probe> ]
[ http_file: <http_file_probe> ]
```
### <tls_config>
@@ -339,6 +379,13 @@ target: <string>
[ kubeconfig: <string> ]
```
### <http_file_probe>
```
# HTTP proxy server to use to connect to the targets.
[ proxy_url: <string> ]
```
## Example Queries
Certificates that expire within 7 days:

View File

@@ -17,22 +17,25 @@ var (
DefaultConfig = &Config{
DefaultModule: "tcp",
Modules: map[string]Module{
"tcp": Module{
"tcp": {
Prober: "tcp",
},
"http": Module{
"http": {
Prober: "https",
},
"https": Module{
"https": {
Prober: "https",
},
"file": Module{
"file": {
Prober: "file",
},
"kubernetes": Module{
"http_file": {
Prober: "http_file",
},
"kubernetes": {
Prober: "kubernetes",
},
"kubeconfig": Module{
"kubeconfig": {
Prober: "kubeconfig",
},
},
@@ -73,6 +76,7 @@ type Module struct {
HTTPS HTTPSProbe `yaml:"https,omitempty"`
TCP TCPProbe `yaml:"tcp,omitempty"`
Kubernetes KubernetesProbe `yaml:"kubernetes,omitempty"`
HTTPFile HTTPFileProbe `yaml:"http_file,omitempty"`
}
// TLSConfig is a superset of config.TLSConfig that supports TLS renegotiation
@@ -142,6 +146,11 @@ type KubernetesProbe struct {
Kubeconfig string `yaml:"kubeconfig,omitempty"`
}
// HTTPFileProbe configures a http_file probe
type HTTPFileProbe struct {
ProxyURL URL `yaml:"proxy_url,omitempty"`
}
// URL is a custom URL type that allows validation at configuration load time
type URL struct {
*url.URL

View File

@@ -34,3 +34,18 @@ scrape_configs:
static_configs:
- targets:
- 127.0.0.1:9219
- job_name: 'ssl-http-files'
metrics_path: /probe
params:
module: ["http_file"]
static_configs:
- targets:
- 'https://www.paypalobjects.com/marketing/web/logos/paypal_com.pem'
- 'https://d3frv9g52qce38.cloudfront.net/amazondefault/amazon_web_services_inc_2024.pem'
relabel_configs:
- source_labels: [__address__]
target_label: __param_target
- source_labels: [__param_target]
target_label: instance
- target_label: __address__
replacement: 127.0.0.1:9219

View File

@@ -38,6 +38,12 @@ modules:
file_ca_certificates:
prober: file
target: /etc/ssl/certs/ca-certificates.crt
http_file:
prober: http_file
http_file_proxy:
prober: http_file
http_file:
proxy_url: "socks5://localhost:8123"
kubernetes:
prober: kubernetes
kubernetes_kubeconfig:

60
prober/http_file.go Normal file
View File

@@ -0,0 +1,60 @@
package prober
import (
"context"
"fmt"
"io"
"net/http"
"github.com/go-kit/log"
"github.com/prometheus/client_golang/prometheus"
"github.com/ribbybibby/ssl_exporter/v2/config"
)
// ProbeHTTPFile collects certificate metrics from a remote file via http
func ProbeHTTPFile(ctx context.Context, logger log.Logger, target string, module config.Module, registry *prometheus.Registry) error {
proxy := http.ProxyFromEnvironment
if module.HTTPFile.ProxyURL.URL != nil {
proxy = http.ProxyURL(module.HTTPFile.ProxyURL.URL)
}
tlsConfig, err := config.NewTLSConfig(&module.TLSConfig)
if err != nil {
return fmt.Errorf("creating TLS config: %w", err)
}
client := &http.Client{
Transport: &http.Transport{
TLSClientConfig: tlsConfig,
Proxy: proxy,
DisableKeepAlives: true,
},
}
req, err := http.NewRequestWithContext(ctx, http.MethodGet, target, nil)
if err != nil {
return fmt.Errorf("creating http request: %w", err)
}
req.Header.Set("User-Agent", userAgent)
resp, err := client.Do(req)
if err != nil {
return fmt.Errorf("making http request: %w", err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return fmt.Errorf("unexpected response code: %d", resp.StatusCode)
}
body, err := io.ReadAll(resp.Body)
if err != nil {
return fmt.Errorf("reading response body: %w", err)
}
certs, err := decodeCertificates(body)
if err != nil {
return fmt.Errorf("decoding certificates from response body: %w", err)
}
return collectCertificateMetrics(certs, registry)
}

112
prober/http_file_test.go Normal file
View File

@@ -0,0 +1,112 @@
package prober
import (
"context"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/prometheus/client_golang/prometheus"
"github.com/ribbybibby/ssl_exporter/v2/config"
"github.com/ribbybibby/ssl_exporter/v2/test"
)
func TestProbeHTTPFile(t *testing.T) {
testcertPEM, _ := test.GenerateTestCertificate(time.Now().AddDate(0, 0, 1))
server := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Write(testcertPEM)
}))
server.Start()
defer server.Close()
registry := prometheus.NewRegistry()
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
if err := ProbeHTTPFile(ctx, newTestLogger(), server.URL+"/file", config.Module{}, registry); err != nil {
t.Fatalf("error: %s", err)
}
cert, err := newCertificate(testcertPEM)
if err != nil {
t.Fatal(err)
}
checkCertificateMetrics(cert, registry, t)
}
func TestProbeHTTPFile_NotCertificate(t *testing.T) {
server := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("foobar"))
}))
server.Start()
defer server.Close()
registry := prometheus.NewRegistry()
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
if err := ProbeHTTPFile(ctx, newTestLogger(), server.URL+"/file", config.Module{}, registry); err == nil {
t.Errorf("expected error but got nil")
}
}
func TestProbeHTTPFile_NotFound(t *testing.T) {
server := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusNotFound)
}))
server.Start()
defer server.Close()
registry := prometheus.NewRegistry()
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
if err := ProbeHTTPFile(ctx, newTestLogger(), server.URL+"/file", config.Module{}, registry); err == nil {
t.Errorf("expected error but got nil")
}
}
func TestProbeHTTPFileHTTPS(t *testing.T) {
server, certPEM, _, caFile, teardown, err := test.SetupHTTPSServer()
if err != nil {
t.Fatalf(err.Error())
}
defer teardown()
server.Config.Handler = http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Write(certPEM)
})
server.StartTLS()
defer server.Close()
module := config.Module{
TLSConfig: config.TLSConfig{
CAFile: caFile,
InsecureSkipVerify: false,
},
}
registry := prometheus.NewRegistry()
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
if err := ProbeHTTPFile(ctx, newTestLogger(), server.URL+"/file", module, registry); err != nil {
t.Fatalf("error: %s", err)
}
cert, err := newCertificate(certPEM)
if err != nil {
t.Fatal(err)
}
checkCertificateMetrics(cert, registry, t)
}

View File

@@ -12,9 +12,12 @@ import (
"github.com/go-kit/log"
"github.com/go-kit/log/level"
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/common/version"
"github.com/ribbybibby/ssl_exporter/v2/config"
)
var userAgent = fmt.Sprintf("SSLExporter/%s", version.Version)
// ProbeHTTPS performs a https probe
func ProbeHTTPS(ctx context.Context, logger log.Logger, target string, module config.Module, registry *prometheus.Registry) error {
tlsConfig, err := newTLSConfig("", registry, &module.TLSConfig)
@@ -57,6 +60,7 @@ func ProbeHTTPS(ctx context.Context, logger log.Logger, target string, module co
return err
}
request = request.WithContext(ctx)
request.Header.Set("User-Agent", userAgent)
resp, err := client.Do(request)
if err != nil {
return err

View File

@@ -3,6 +3,7 @@ package prober
import (
"context"
"fmt"
"regexp"
"strings"
"github.com/bmatcuk/doublestar/v2"
@@ -22,6 +23,8 @@ var (
// ErrKubeBadTarget is returned when the target doesn't match the
// expected form for the kubernetes prober
ErrKubeBadTarget = fmt.Errorf("Target secret must be provided in the form: <namespace>/<name>")
globPattern = regexp.MustCompile(`^.*(\*|\?|\{|\}|\[|\])+.*$`)
)
// ProbeKubernetes collects certificate metrics from kubernetes.io/tls Secrets
@@ -43,8 +46,15 @@ func probeKubernetes(ctx context.Context, target string, module config.Module, r
ns := parts[0]
name := parts[1]
// If the namespace contains a glob pattern then we need to filter on
// all the secrets in the cluster
selector := ns
if globPattern.MatchString(ns) {
selector = ""
}
var tlsSecrets []v1.Secret
secrets, err := client.CoreV1().Secrets("").List(ctx, metav1.ListOptions{FieldSelector: "type=kubernetes.io/tls"})
secrets, err := client.CoreV1().Secrets(selector).List(ctx, metav1.ListOptions{FieldSelector: "type=kubernetes.io/tls"})
if err != nil {
return err
}

View File

@@ -15,6 +15,7 @@ var (
"http": ProbeHTTPS,
"tcp": ProbeTCP,
"file": ProbeFile,
"http_file": ProbeHTTPFile,
"kubernetes": ProbeKubernetes,
"kubeconfig": ProbeKubeconfig,
}

View File

@@ -57,7 +57,7 @@ func contains(certs []*x509.Certificate, cert *x509.Certificate) bool {
func decodeCertificates(data []byte) ([]*x509.Certificate, error) {
var certs []*x509.Certificate
for block, rest := pem.Decode(data); block != nil; block, rest = pem.Decode(rest) {
if block.Type == "CERTIFICATE" {
if block.Type == "CERTIFICATE" || block.Type == "TRUSTED CERTIFICATE" {
cert, err := x509.ParseCertificate(block.Bytes)
if err != nil {
return certs, err

86
prober/tls_test.go Normal file
View File

@@ -0,0 +1,86 @@
package prober
import (
"testing"
)
func TestDecodeCertificates(t *testing.T) {
data := []byte(`
-----BEGIN CERTIFICATE-----
MIIFszCCA5ugAwIBAgIUdpzowWDU/AI7QBhLSRB9DPqpWvcwDQYJKoZIhvcNAQEL
BQAwaTELMAkGA1UEBhMCWFgxEjAQBgNVBAgMCVN0YXRlTmFtZTERMA8GA1UEBwwI
Q2l0eU5hbWUxFDASBgNVBAoMC0NvbXBhbnlOYW1lMQwwCgYDVQQLDANGb28xDzAN
BgNVBAMMBkZvb2JhcjAeFw0yNDA0MjgxNjIzMzNaFw0zNDA0MjYxNjIzMzNaMGkx
CzAJBgNVBAYTAlhYMRIwEAYDVQQIDAlTdGF0ZU5hbWUxETAPBgNVBAcMCENpdHlO
YW1lMRQwEgYDVQQKDAtDb21wYW55TmFtZTEMMAoGA1UECwwDRm9vMQ8wDQYDVQQD
DAZGb29iYXIwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQCmxZBW/Ays
VBxt7jJQeTrPdLQpUdxnGVXOa4M54FHWA2DwwmZ2DZth5Eioq1wC9WCrByWkd8px
mvU0XDUT5ESceEKcwmDhKgYHAcJ4qXEyk1jYuBy6zw95cmV2BiTf0Xoo/8JxiQR4
YBd7Tbm52eV5Hw5oaqgVEdaOCVMnO8S57AuQGfeC5AO18ty0cZ7mKsXQje2celMH
QipujXrhRwdLBgu6FISTuS0XtqiuJnp+vllMjiTMF/uCmJUTUCpmayWSpM1FRpKe
lM8B9666uuEmVJ4V5gzy4Oe0i5Apfh4qXX6pj+Y/oeOfXRc3NfYqcIJ2hm82ghJL
5Kt6FZ9fkQ6pyk7nXMAOaf8WX+JkhqaWvlzTme8Z+6DBXLPfyyoi3VR2kG3lRida
qfyh2EPvEXip810s4f8EOHi+sehmjWLbsgn2HAHQmE7zYTEMp2Xsh0M7lGfUiMfP
P1RU/RNDkZK59yXsG9RmoDMD03qI8M4990TL7BW0FQZvBg9Rfr3KgFpWngsn84mN
l6/MKyc5X1e+RGyYJPbi0S1j+fhBeNzFqQPFZZUnWIPTxVdqF91SgY9EwE6MHuD6
t7G+eANWWQkolvwACMT+0GqiRMVHVWeIqF2SQ7Dx2tiNXjE9rSX3lMpMNt4PvWcM
PGuao3FMqXts0j2L8FSljcVU/hiLOGC2mwIDAQABo1MwUTAdBgNVHQ4EFgQU+231
i0rY0CwQ0JwT2EUX4cq81LIwHwYDVR0jBBgwFoAU+231i0rY0CwQ0JwT2EUX4cq8
1LIwDwYDVR0TAQH/BAUwAwEB/zANBgkqhkiG9w0BAQsFAAOCAgEAVRqUVxovcZ9C
BYUUhvyT8BMAC6PS8U59BoDhnbrp1JDiPDbMnWlRp+450kZlEkSdN9PRLGWoGi84
QqT7RIry/jA4z62B6N+IDDDcuWWstDTTC/gRA46eeHGRnIz7GJeseqrxb9z8KoO3
/c8Pdb4lu/cfG5/m2X8hkYzjLZrIQ8qz53nUAag4uael19uy3HPQ3EEr936y+vQW
rH8itgQ+5kOTr9d3ihcTSwTJGTyWq7j3T0xPu9tCzoPragpEDjoUCqCjU7uv6Tdq
UhaQ8vneimSf9VdbDfEuEU9S2Xbdg6e+BsdV9uMeWkhtD1WgtSLHcliYktwnh/kh
r4vQG86xn+LDxTrdTssjt+UmJnXzRiGOEZCDz7kBchYLiWSQn9tqcn28KK2/YobD
AghlR24hUL1uslJOjLFc4BwLmlv/4Iy+b/3iQWY+yoban/OLZo6gCZx/4Nvg5R9e
tcuxgn5Jhm2T/e8REucXQfiqKgxQVibFmqXeLH3Yj7ussA5t/ozo4VuypXUnnil0
hB+PQsViaHFId1LyBKFwMoA6JzlNle6cbWVtXJswffkjk0UUr26SLHMV87lrk2kn
IQ7ROZcT8K7gzzTxuiC2g6npmG1fDfgmgJeS7IqgiFcRTKa0hj6bvc/WoHoCpLNs
v5KXxrGlqibeNjyc3Wng6S0Kpg6YNqU=
-----END CERTIFICATE-----
-----BEGIN TRUSTED CERTIFICATE-----
MIIF7zCCA9egAwIBAgIUZXsnB0gxoPSFaUjnQeLwTW+sqbYwDQYJKoZIhvcNAQEL
BQAwgYYxCzAJBgNVBAYTAlhYMRIwEAYDVQQIDAlTdGF0ZU5hbWUxETAPBgNVBAcM
CENpdHlOYW1lMRQwEgYDVQQKDAtDb21wYW55TmFtZTEbMBkGA1UECwwSQ29tcGFu
eVNlY3Rpb25OYW1lMR0wGwYDVQQDDBRDb21tb25OYW1lT3JIb3N0bmFtZTAeFw0y
NDA0MjgxNjE3MzZaFw0zNDA0MjYxNjE3MzZaMIGGMQswCQYDVQQGEwJYWDESMBAG
A1UECAwJU3RhdGVOYW1lMREwDwYDVQQHDAhDaXR5TmFtZTEUMBIGA1UECgwLQ29t
cGFueU5hbWUxGzAZBgNVBAsMEkNvbXBhbnlTZWN0aW9uTmFtZTEdMBsGA1UEAwwU
Q29tbW9uTmFtZU9ySG9zdG5hbWUwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIK
AoICAQDJ+g1z+nZFj0DFc+wV1AH4giAqDFhwx25yziaQJnHkoXpllN0bRpr50Fhx
QsMuMqjMCM0eFPo7kDWSrAUusdh3j1jdF7KidZRZdyUgioQvbmNtqtwdU/38Gq0s
yTspOTZTWziIMFsLYxvFb0Ia8dwQFvyDB3pM1/eXFJMr2Cz/nF4/g4IUFEtPY2pe
4sQDMhszvROMfI5LB4SmGiim/zUS7+ZDEQigf10/CrbLntjXHiFy+V0hcrDQXkuY
gXU9HsiwRpCx+GRqcI6SWPIZ2rYa7goAVBENv5KWFXdWaOFCpi55XldYYzDeaXWa
On9EOjyylLbJ2pv/yhpt/VYDC6Lsqki1XcU/zeETcBRlqzCwZhMqFErSPRBGxaTN
lEGKNS2qQnE7I7bmUksRcCxA0WIn4V2xxxGHkLFnEsCipFGgMm7LaGSukYZGQ2MF
0ELqaaRMaTBfvPhFRtolkFChm7JQlCrk9j47b07OTMj4KUoBkhYPNoK26sHjvKue
Iks7BTGy8NdZk3jVDNEgGwngj+bNyJcjDvq+cHhWhS/N8c8Tg2tFb3ZOnwbyFPLf
1DdsP/SW6TybMc6NtpWvmSCFMvxqRpr+7LscpBGThepAUaoPkATnT6UdYGqr+l5N
z1+cZEuHBF7Jk5xThXCCFwuBC/KREvRuU2LyAhm9RV1ci8XyYwIDAQABo1MwUTAd
BgNVHQ4EFgQUa1FF7jssOpGY0UY1qbFQH83rxOQwHwYDVR0jBBgwFoAUa1FF7jss
OpGY0UY1qbFQH83rxOQwDwYDVR0TAQH/BAUwAwEB/zANBgkqhkiG9w0BAQsFAAOC
AgEAhRPDPmVetjE5w/igfnGiNUKhQiqB3E6yT1wMwF5LbNIC+h8E8WqF0mzF3XQC
t9+Eib+mFptHGfRGdA8JmJ79xBgch6JLagw8Ot6hA1oEqWy/PAzcFLemccvhonhZ
+N7umRO27agIYVJ71mxlS7rD1SItBKZ+g4/Lt3sr/iDM0kIhWjWdQiYgJMNhH2Mt
XAd53tmPSom1Vsca1ZorB0HJNIw8RB7QYwceAi0xk0Tui7Z6pHg8p99XHCK/2vD+
+u5a1nHjtrAvLdNti8o82EUScK4j91CODCm5bv2KDMIUUv+1O83UMicK8DLSIi0P
1acuIdbRrY5JiYewCC+NFCfODZT6y00QL7aF9cbT+ZySiYtpBUiLFlIZCw4dcJ6r
yYJca1gv+kqZugdgLD7nXWIzXgfreQ+Q/BiVouXroXaWqU/9dqzGw4LlPSpyqI9o
nFdu970j4BvVc809i1aNo3odGz9yRx7wOOMyiyHoRpYenlf3hFbqJHS2FUgJ/Ajl
TC1joWDPdwdqQY56WOpk2LMK1KJuMYQEEHb3Ib0ZaxXAT6Nd12VashG5gYhw9vGn
aZ3nCnZFahgB2tB5DtkK+p0PQVpZOv6/0CKSsfQkBFLTjeVIRnlL/UlK3VlwHH9h
0LfsLFabedOS9R1XIaQ7aKXPK4tKmNb5H/ONzH2zxU5Oqoo=
-----END TRUSTED CERTIFICATE-----
`)
certs, err := decodeCertificates(data)
if err != nil {
t.Errorf("unexpected error: %s", err)
}
if len(certs) != 2 {
t.Errorf("unexpected number of certs: %d", len(certs))
}
}