Help Manual - Google Cloud Storage Connection Manager

Documentation on SSIS Google Cloud Storage Connection Manager component

Using the Google Cloud Storage Connection Manager

The Google Cloud Storage Connection Manager is an SSIS connection manager component that can be used to establish connections with Google Cloud Storage.

To add a Google Cloud Storage connection to your SSIS package, right-click the Connection Manager area in your Visual Studio project, and choose "New Connection..." from the context menu. You will be prompted the "Add SSIS Connection Manager" window. Select the "Google Cloud Storage (KingswaySoft)" item to add the new Google Cloud Storage Connection Manager.

new connection

add SSIS Google Cloud Storage connection manager

The Google Cloud Storage Connection Manager contains the following two pages which configure how you want to connect to Google Cloud Storage.

  • General
  • Advanced Settings

General Page

The General page on the Google Cloud Storage Connection Manager allows you to specify general settings for the connection.

SSIS Google Cloud Storage Connection Manager

Project Id

This option allows you to specify the Project Id which is a unique identifier for your Google project.

Authentication Mode

The Authentication Mode option allows you to specify how you want to authenticate to your Google Service. Available options are:

  • Authorization Code
  • Service Account (server-to-server authentication)
Token Location (Available only when Authorization Code is selected)

Google Cloud Storage retrieves tokens from the Google app once you give authorization. The tokens are stored in Windows Registry so that once they expire, our component will automatically request new tokens from Google and replace the existing tokens with the new ones in the Registry to minimize user effort and avoid service interruption for scheduled jobs. You have two options to choose where in the Registry you would like to store these tokens:

  • Local Machine registry hive – HKEY_LOCAL_MACHINE
  • Current User hive – HKEY_CURRENT_USER
App Type (Available only when Authorization Code is selected)

The selected App Type that was chosen when creating the Google token.

Account (Available only when Authorization Code is selected)

The selected available Google Account Number and Name.

Get New (Available only when Authorization Code is selected)

This button will open a dialog that allows you to log in to the service endpoint and authorize your app to generate a new token.

SSIS Google Cloud Storage Connection

  • Google Service: The Google Service you are trying to connect to.
  • Token Location: The windows registry location where the new token will be saved.
  • Authorization Type: The App which will be used to connect to your Google Service. Available options are:
    • KingswaySoft: The Connection Manager will use the Client Id and Secret that is provided by KingswaySoft for your convenience when setting up the connection, it should NEVER be used for production purposes.
    • My Own App: Selecting this option will allow you to use your own Client ID and Secret to connect to Google Services.
  • Client Id: The Client Id option allows you to specify the unique ID which identifies the application making the request.
  • Client Secret: The Client Secret option allows you to specify the secret belonging to your Google Service App ID.
  • Redirect Url: The Redirect Url option allows you to specify the Redirect Url which was used during the Google App creation in order to complete the authentication process.
  • Generate Token File in App: This button completes the entire OAuth authentication process inside of the toolkit. All you need to do is log in to the service endpoint and authorize our app to generate your token.
  • Generate Token in Browser: This button completes the OAuth authentication using your default browser. After you click this button simply follow the steps in the browser to generate your token.

Warning: By default you are using the Google Cloud Storage API authorization information (Client Id and Secret) provided by KingswaySoft, which is provided for your convenience when setting up the connection, and it should NEVER be used for production purposes. To obtain your own Google Cloud Storage API authorization credentials for product use, please follow the instructions at the link below to create your own application registration with Google: https://cloud.google.com/storage/docs/json_api/v1/how-tos/authorizing#OAuth2Authorizing

Key File Type (Available only when Service Account is selected) (since v21.1)

Specify the private key file format that was downloaded from Google, available options are:

  • JSON
  • P12

In order to work with the Service Account option, a service account would need to be created in Google Console to generate a certificate key. The following steps can be used for that:

  • Log in to the Google Cloud Platform Console

  • Navigate to IAM & Admin Console.

  • Choose "Service Accounts" from the list, and then click on "Create Service Account".

    Create Service Account

  • Provide the "Service Account Name", and the "Service Account Id" would be auto-populated based on the Name.

  • Give permissions if required by switching to the "Permissions" tab.

  • Choose the "Keys" tab, and then click on "Add Key" and select "Create New Key".

    Create New Key

  • From the window that pops up, download the private key file.

Service Account (Available only when Service Account is selected)

The Service Account option allows you to specify the account which will be used to connect to your Google service.

Certificate Location (Available only when Service Account is selected)

Allows you to specify the location of the certificate that will be used for authentication. There are two options available:

  • Store
  • File System
Certificate Thumbprint (Available only when Service Account is selected)

This option allows you to specify the thumbprint of the selected certificate in order to authenticate using your Google Service. Account. Available only when Store is selected as Certificate Location.

Certificate File Path

Provide the path to the certificate file to use for authentication. Click the ellipses button to browse the file system for the certificate. This option is available when you choose File System as Certificate Location.

Note: The component supports Azure Blob Shared Access Signature (SAS) URL in the certificate path.

Certificate File Password

Provide the password to the certificate file. This option is available when you choose File System as the Certificate Location.

Test Connection

After all the connection information has been provided, you may click the Test Connection button to test if the connection settings entered are valid.

Advanced Settings Page

The Advanced Settings page of Google Cloud Storage Connection Manager allows you to specify some advanced settings of the connection.

SSIS Google Cloud Storage Connection Manager - Advanced settings

Proxy Server Settings
Proxy Mode

The Proxy Mode option allows you to specify how you want to configure the proxy server setting. There are three options available.

  • No Proxy
  • Auto-detect (Using system-configured proxy)
  • Manual
Proxy Server

Using the Proxy Server option, you can provide a proxy server to connect to Google Cloud Storage.

Port

The Port option allows you to specify the port number of the proxy server for the connection.

Username (Proxy Server Authentication)

The Username option (under Proxy Server Authentication) allows you to specify the proxy user account.

Password (Proxy Server Authentication)

The Password option (under Proxy Server Authentication) allows you to specify the proxy user's password.

Note: The Proxy Password is not included in the connection manager's ConnectionString property by default. This is by design for security reasons. However, you can include it in your ConnectionString if you want to parameterize your connection manager. The format would be ProxyPassword=myProxyPassword; (make sure you have a semicolon as the last character). It can be anywhere in the ConnectionString.

Misc
Upload Chunk Size (in MB)

The UploadChunk Size option allows you to specify the chunk size to use when sending web requests. The default value is 0 MB which means the chunk size will not be used.

Download Chunk Size (in MB)(since v22.1)

Specify the Chunk Size to download large files from Google Cloud Storage in parts.

Timeout (secs)

The Timeout (secs) option allows you to specify a timeout value in seconds for the connection. The default value is 120 seconds.

API Throttling Rate

The API Throttling Rate option will limit the number of requests that can be sent per second. This rate is set to 10 requests per second limit, by default, to respect the Google Cloud Storage API limits.

Retry on Intermittent Errors

This is an option designed to help recover from possible intermittent outages or disruption of service. It prevents the integration process from stopping due to temporary issues. Enabling this option will allow service calls to be retried upon certain types of failure. A service call may be retried up to 3 times before an exception is fired. Retries occur after 0 seconds, 15 seconds, and 60 seconds.

Warning: We have designed our retry feature carefully such that the retry should only occur when it is deemed safe to do so; however, in some occasions, such retry service calls could result in the creation of duplicate data.