Execute a Function

Execute a Function PRODUCT     AVGIDEA, INC. 

You can execute a function on a file stored in a pre-configured Storage datasource. It is also possible to check the log output from the function.

1. From the ADP console, select Storage -> Storage data source, and click the EXECUTE menu from the action button of the corresponding file.

* The maximum execution time for a function is 30 minutes, including data and custom package loading time.

2. Select the created function and the cluster where the function will be executed, and click the EXECUTE button.

Parameters : Pass parameters to the function in JSON format

Notification : Receive notification by email when a function finishes executing

3. Click on Option and you can also specify execution options

Retry: Number of times to re-execute the function if it fails to execute (default 0)

In-place file: Replaces the input file with the output file when a File function is executed (default false)

All files: Execute Directory functions with all files under the Storage Data Source as the target files to be processed

File open: Directory Stores the files argument with the corresponding files open when the function is executed (default true)

Input Encoding: Specify the encoding of input files to be opened by Directory and File functions (default utf-8), conforming to Python encoding

Output Encoding: Specify the encoding of the output file to be opened by the Directory and File functions (default utf-8), conforming to Python encoding



4. In the case of Creating a Function : Line, if you execute PREVIEW on the output file (data/small.csv.out), you will see that only the second column is stored.

Checking the execution log

If the function is outputting logs, you can check the execution log of the function.

1. Log in to the ADP console, select the "Function" menu, and click the "LOGS" menu of the corresponding execution log from Executions.

2. You can check the list of logs output from the function.


* Function execution logs can also be viewed from Storage.