Setting Up an Elasticsearch REST Connection Manager
The REST Connection Manager is an SSIS connection manager that can be used to establish connections and facilitate data integration with REST based APIs and services such as Elasticsearch, among many more.
To add a new connection, right-click the Connection Manager area in your Visual Studio project, and choose "New Connection..." from the context menu. You will be prompted the "Add SSIS Connection Manager" window. Select the "REST (KingswaySoft)" item to add a new REST Connection Manager.
The REST Connection Manager contains 2 pages of configuration.
- General
- Advanced Settings
General page
The General page allows you to specify connection properties and login credentials for Elasticsearch REST service:
- Service Name
-
The REST service that will be connected to.
- Base URL
-
The Base URL of the Elasticsearch instance that will be connected to.
- API Throttle Rate
-
The API Throttling Rate will limit the number of requests that can be sent per minute.
- Authentication Mode
-
The Authentication Mode option allows you to specify the authentication mode when working with the Elasticsearch service. Available options are:
- None
- API Key
- Basic
- Client Credentials
- OpenID Connect
- Delegate PKI
- Password
- AWS Signature
- API Key ID (API Key Authentication Mode)
-
This option allows you to enter your own API token which is going to be used to connect to your Elasticsearch instance.
- API Key (API Key Authentication Mode)
-
This option allows you to enter your own API token which is going to be used to connect to your Elasticsearch instance.
- Show API Key (API Key Authentication Mode)
-
Clicking on this button allows you to see or copy the token in the 'API Key' field.
- Generate API Key (API Key Authentication Mode)
-
This option allows you to enter a Username and a Password in order to generate a new API Key.
- Username (Basic, Client Credentials, OpenID Connect, Delegate PKI, Password Authentication Modes)
-
This option allows you to specify the Username of your Elasticsearch account.
- Password (Basic, Client Credentials, OpenID Connect, Delegate PKI, Password Authentication Modes)
-
This option allows you to specify the user's password
- Realm (OpenID Connect Authentication Mode)
-
This option allows you to specify the Realm to connect to.
- lss (OpenID Connect Authentication Mode)
-
This option allows you to specify the LSS parameter of your connection.
- Certificate Location (Delegate PKI Authentication Mode)
-
Allows you to specify the location of the certificate that will be used. There are two options available:
- Store
- File System
- Certificate Thumbprint (Delegate PKI Authentication Mode)
-
Allows you to specify the thumbprint of the client certificate from the Certificate Store. Click the ellipses button to browse and select a certificate for the user's store. This option is available when you choose Store as Certificate Location.
- Path to Certificate (Delegate PKI Authentication Mode)
-
Provide the path to the certificate file to use for authentication. Click the ellipses button to browse their file system for the certificate. This option is available when you choose File System as Certificate Location.
- Certificate Password (Delegate PKI Authentication Mode)
-
Provide the password to the certificate file. This option is available when you choose File System as the Certificate Location.
- Access Key (AWS Signature Authentication Mode)
-
This option allows you to specify the Access Key which is going to be used for your connection.
- Access Secret (AWS Signature Authentication Mode)
-
This option allows you to specify the secret of your Access Key.
- AWS Region (AWS Signature Authentication Mode)
-
This option allows you to specify the AWS Region of your instance.
- Session Token (AWS Signature Authentication Mode)
-
This option allows you to specify the Session Token which is going to be used for connecting.
- Test Connection
-
After all the connection information has been provided, you may click the "Test Connection" button to test if the user credentials entered can connect to the selected REST service.
Advanced Settings page
- Proxy Mode
-
The Proxy Mode option allows you to specify how you want to configure the proxy server setting. There are three options available.
- No Proxy
- Auto-detect (Using system-configured proxy)
- Manual
- Proxy Server
-
Using the Proxy Server option allows you to specify the name of the proxy server for the connection.
- Port
-
The Port option allows you to specify the port number of the proxy server for the connection.
- Username (Proxy Server Authentication)
-
The Username option (under Proxy Server Authentication) allows you to specify the proxy user account.
- Password (Proxy Server Authentication)
-
The Password option (under Proxy Server Authentication) allows you to specify the proxy user's password.
Note: The Proxy Password is not included in the connection manager's ConnectionString property by default. This is by design for security reasons. However, you can include it in your ConnectionString if you want to parameterize your connection manager. The format would be ProxyPassword=myProxyPassword; (make sure you have a semicolon as the last character). It can be anywhere in the ConnectionString.
- Timeout
-
The number of seconds requests should wait before timing out.
- Retry on Intermittent Errors
-
The retry on intermittent errors option determines if requests will be retried when there is an error. If this option is checked requests will be retried up to 3 times.
- Ignore Certificate Errors
-
This option can be used to ignore those SSL certificate errors when connecting to Slack servers.
Warning: Enabling the "Ignore Certificate Errors" option is generally NOT recommended, particularly for production instances. Unless there is a strong reason to believe the connection is secure - such as the network communication is only happening in an internal infrastructure, this option should be unchecked for best security.