Using the OneLake Connection Manager
The OneLake Connection Manager is an SSIS connection manager component that can be used to establish connections with Microsoft OneLake storage instance.
To add a OneLake connection to your SSIS package, right-click the Connection Manager area in your Visual Studio project, and choose "New Connection..." from the context menu. You will be prompted the "Add SSIS Connection Manager" window. Select the "OneLake" item to add the new OneLake Connection Manager.
The OneLake Connection Manager contains the following two pages which configure how you want to connect to the instance.
- General
- Advanced Settings
General Page
The General page on the OneLake Connection Manager allows you to specify general settings for the connection.
- Workspace
-
- Workspace
-
This option allows you to specify the workspace name for the instance.
- Data Region
-
Choose the data region from the list.
- Data Storage Endpoint
-
This field gets populated automatically based on the Data Region that is chosen.
- Generate Token File...
-
Click this button to open up the OAuth token generator. All you need to do is enter the details, log into the service endpoint, and authorize our app to generate your token.
-
-
- Tenant ID
-
The Tenant ID option allows you to specify the unique ID that identifies the tenant you are connecting to.
- Client ID
-
The Client ID option allows you to specify the unique ID that identifies the application making the request.
- Client Secret
-
The Client Secret option allows you to specify the client secret belonging to your app.
- Redirect URL
-
The Redirect URL option allows you to specify the Redirect URL to complete the authentication process.
- Use default browser to sign in
-
The checkbox lets you pick between either opening the default browser, or using the In app feature to sign in and authorize the OAuth App.
- Path to Token File
-
The path to the token file on the file system.
- Token File Password
-
The password to the token file.
-
- Misc
-
- Bulk Download Behavior
-
The Bulk Download Behavior can be chosen from the below three options. Note that the component will not take any special measures during download.
-
Allow external file modifications
-
Fail on external file modifications
-
Prevent external file modifications
-
- Upload Chunk Size (in MB)
-
The Upload Chunk Size option allows you to specify the size of the file content to be divided to upload large files sequentially.
- Download Chunk Size (in MB)
-
Specify the Chunk Size to download large files from OneLake in parts.
- Timeout (secs)
-
The Timeout (secs) option allows you to specify a timeout value in seconds for the connection. The default value is 120 seconds.
- Retry on Intermittent Errors
-
This is an option designed to help recover from possible intermittent outages or disruption of service so the integration does not have to be stopped because of such temporary issues. Enabling this option will allow service calls to be retried upon certain types of failure. A service call may be retried up to 3 times before an exception is fired. Retries occur after 0 seconds, 15 seconds, and 60 seconds. Warning: although we have carefully designed this feature so that such retries should only happen when it is deemed to be safe to do so, in some extreme occasions, such retried service calls could result in the creation of duplicate data.
- Test Connection
-
After all the connection information has been provided, you may click the Test Connection button to test if the connection settings entered are valid.
Advanced Settings Page
The Advanced Settings page on the OneLake Connection Manager allows you to specify some advanced and optional settings for the connection.
- Proxy Server Settings
-
- Proxy Mode
-
The Proxy Mode option allows you to specify how you want to configure the proxy server setting. There are three options available.
- No Proxy
- Auto-detect (Using system-configured proxy)
- Manual
- Proxy Server
-
Using the Proxy Server option allows you to specify the name of the proxy server for the connection.
- Port
-
The Port option allows you to specify the port number of the proxy server for the connection.
- Username (Proxy Server Authentication)
-
The Username option (under Proxy Server Authentication) allows you to specify the proxy user account.
- Password (Proxy Server Authentication)
-
The Password option (under Proxy Server Authentication) allows you to specify the proxy user's password.
Note: The Proxy Password is not included in the connection manager's ConnectionString property by default. This is by design for security reasons. However, you can include it in your ConnectionString if you want to parameterize your connection manager. The format would be ProxyPassword=myProxyPassword; (make sure you have a semicolon as the last character). It can be anywhere in the ConnectionString.