Skip to main content

Profiler

Overview​

The Profiler helps you understand and optimize the CPU performance of your processing rules and flow configurations. By continuously capturing detailed execution data from the processing pipeline, it reveals which rules consume the most CPU time and helps you identify optimization opportunities in your data processing workflows.

What Does the Profiler Show?​

The Profiler focuses on CPU consumption during rule execution within your flow configurations. It answers questions like:

  • Which rules are most CPU-intensive?
  • How much CPU time does a specific flow configuration consume?
  • Which functions within a rule are causing performance bottlenecks?
  • How does rule performance change over time?

Key Features​

  • Rule-Level Analysis: See CPU consumption broken down by individual rules
  • Flow Configuration Insights: Understand the performance characteristics of entire flow configurations
  • Flame Graph Visualization: Interactive flame graphs show the call stack and execution path of your rules
  • Time-Based Analysis: View rule performance over different time ranges (15 minutes to 7 days)
  • Datasource-Specific Profiling: Compare how rules perform when processing different datasources
  • Timeline View: Track CPU consumption patterns to identify peak processing times

Permissions and Access​

Required Permissions​

To access the Profiler, you need read permissions on the profiling resource for your namespace. Contact the support team if you don't have access to the Profiler.

Namespace Scope​

The Profiler shows data only for namespaces you have access to:

  • You can only view profiling data for rules and flows within your authorized namespaces
  • When filtering by datasource or flow configuration, only resources from your accessible namespaces will appear
  • If you have access to multiple namespaces, you'll see profiling data from all of them (filtered by your active namespace selection)

Accessing the Profiler​

The Profiler is available in the Observability section of the platform:

  1. Navigate to Observability in the main menu
  2. Select Profiling from the submenu
  3. The Profiler dashboard will load, showing CPU data for rule executions across your flows

Analyzing Rule and Flow Performance​

Selecting a Time Range​

At the top-right of the Profiler interface, choose a time range to analyze rule performance:

  • Last 15 minutes: Check current rule execution performance
  • Last 1 hour: Analyze recent processing runs
  • Last 6 hours: Review rule performance across multiple processing cycles
  • Last 24 hours: Identify daily performance patterns
  • Last 7 days: Compare rule performance over time or different data volumes

Click the time range dropdown and select your desired period, then click Execute Query to load the profiling data.

Filtering by Rules and Flows​

Focus your analysis on specific rules, flow configurations, or datasources:

  1. Click + Add Filter to add filtering criteria
  2. Select a filter type:
    • Rule Name: Analyze a specific rule
    • Flow Config ID: View all rules within a flow configuration
    • Datasource ID: See how rules perform on a particular datasource
    • Sequence ID: Filter by sequence
  3. Select a value from the available options
  4. Click the checkmark (✓) to apply the filter
  5. Click Execute Query to view the filtered results

Tip: Start with a Rule Name filter to investigate specific rule performance, or use Flow Config ID to understand the overall CPU cost of a flow configuration.

Understanding the Timeline​

The timeline chart shows when your rules consumed CPU time:

  • Bar Height: Indicates CPU time consumed during that period (in nanoseconds)
  • Patterns: Recurring peaks may indicate scheduled processing jobs or periodic data ingestion
  • Spikes: Sudden increases might indicate large data volumes or inefficient rule execution

The total CPU time consumed across your selected time range is displayed above the chart.

Reading the Flame Graph​

The flame graph visualizes the execution path and CPU consumption of your rules:

Structure​

  • Width: The width of each box shows the proportion of CPU time consumed by that function
  • Height: Vertical layers represent the call stack - functions at the top are called by functions below them
  • Color: Different colors distinguish between modules, packages, and libraries
  • Labels: Function and method names appear inside the boxes

Rule Execution Path​

The flame graph shows what happens when your rule executes:

  • Rule Entry Points: Look for your rule name near the bottom
  • Data Operations: Boxes showing database queries, API calls, or data transformations
  • Library Functions: Built-in functions and third-party library calls made by your rules
  • Hot Paths: The widest boxes reveal where your rule spends most of its time
  • Click to Zoom: Click any box to focus on that function and everything it calls
  • Reset Zoom: Return to the full view using the Reset Zoom button
  • Hover for Details: Hover over boxes to see exact function names and CPU time

Interpreting Rule Performance​

  • Wide boxes in your rule code: Indicate functions that consume significant CPU time - potential optimization targets
  • Wide boxes in library code: May indicate expensive operations like regex matching, JSON parsing, or mathematical calculations
  • Tall, narrow stacks: Suggest deep recursion or complex nested function calls
  • Many small boxes: Could indicate frequent method calls that add up to significant overhead

CPU Time Metrics​

At the top of the flame graph:

  • Total Samples: The number of times the profiler captured your rule's execution state
  • Total Duration: Aggregate CPU time consumed, displayed in appropriate units (ns, µs, ms, or s)

Higher values indicate more CPU-intensive rule execution or longer processing times.

Understanding Profile Data​

What Gets Profiled?​

The Profiler captures CPU performance data during rule execution:

  • Rule execution: All code executed within your rules
  • Data transformations: Operations that modify or restructure data
  • Validation logic: Checks and validations performed on data
  • Calculations: Mathematical operations, aggregations, and computations
  • External calls: API requests, database queries, and other operations initiated by rules
  • Library functions: Standard and third-party library code called by your rules

How Profiling Works​

The profiling system runs continuously and automatically:

  • Always active: Profiling runs in the background without any manual setup or activation
  • Automatic tagging: Each profile is automatically tagged with rule name, datasource ID, flow configuration, and other context
  • Time-based aggregation: Data is aggregated to show cumulative CPU time over your selected time range

What the Data Shows​

  • Processing time: The Profiler measures actual computation time during rule execution
  • Relative proportions: The width of flame graph boxes accurately represents the proportion of CPU time
  • Function execution paths: The call stack shows which functions call which others during rule execution
  • Performance hot spots: Functions that appear with large width are consuming significant CPU resources

Data Accuracy​

  • Highly accurate: The profiler provides reliable representations of CPU usage patterns
  • Aggregated view: When analyzing longer time ranges, you see the combined CPU usage across all executions of a rule
  • Representative samples: Higher sample counts indicate longer-running or more frequently executed code

Troubleshooting​

Cannot Access the Profiler​

If you cannot see the Profiler option in the Observability menu:

  • Check permissions: You need read permissions on the profiling resource for your namespace
  • Contact administrator: Request profiling access from the support team
  • Verify namespace selection: Ensure you have an active namespace selected

No Data Available for a Rule​

If you see "No profiling data available for the selected criteria" when filtering by a specific rule:

  • Verify the rule executed: Check that the rule has actually processed data during your selected time range
  • Check rule name: Ensure you've entered the exact rule name (case-sensitive)
  • Expand time range: Try a longer time range if the rule executes infrequently
  • Check flow configuration: Verify the rule is active in the flow configuration you expect

Timeline Shows No Activity​

If the timeline is flat with no bars:

  • No data processed: The selected flow/rule may not have processed data during this time
  • Datasource filter too specific: Try removing the datasource filter to see if other datasources show activity
  • Flow not active: Verify that the flow configuration is enabled and receiving data
  • Time range mismatch: Ensure you're looking at a time when you expect processing to occur

Flame Graph is Empty​

If the flame graph area is blank after executing a query:

  • Click Execute Query: Ensure you've clicked Execute Query after setting filters
  • Check timeline first: If the timeline shows no activity, there's no data to display in the flame graph
  • Broaden filters: Remove some filters to see if data exists for broader criteria
  • Verify rule execution: Confirm that your rule actually executed during the selected time range

Can't Find My Rule in the Filter Dropdown​

If your rule name doesn't appear in the Rule Name filter dropdown:

  • Rule hasn't executed yet: Rules only appear in the dropdown after they've executed and generated profile data
  • Try a longer time range: Expand the time range to include when the rule last executed
  • Use a different filter first: Try filtering by Flow Config ID or Datasource ID to narrow the available rules

Flame Graph Shows Only Library Code​

If the flame graph shows mostly library functions without your rule code:

  • Zoom in: Your rule code may be at a deeper level - click on relevant boxes to zoom in
  • Short execution time: Very fast rules may show primarily library overhead
  • Check rule implementation: If you see only library code, your rule might be very thin wrappers around library calls

Unexpected High CPU Usage​

If a rule shows unexpectedly high CPU consumption:

  • Data volume increase: Check if you're processing more data than usual
  • New data characteristics: Different data formats or values might trigger different code paths
  • Rule changes: Recent updates to the rule might have introduced inefficiencies
  • Downstream dependencies: External API slowness can cause CPU consumption while waiting/retrying
  • Compare time periods: Use the 7-day view to see if this is a new pattern

Tips for Effective Profiling​

Start Broad, Then Narrow​

  1. Begin with a Flow Config ID filter to see the overall picture
  2. Identify the most expensive rules in the flow
  3. Filter by specific Rule Name to dive deeper
  4. Analyze the flame graph to find hot spots within that rule

Look for Patterns​

  • Repeated wide boxes: Same function called many times - potential for caching or batching
  • Unexpected library calls: Third-party libraries that might have more efficient alternatives
  • Deep stacks: Complex nesting that might be simplified
  • I/O operations: Database or API calls that could be optimized or parallelized

Know What's Normal​

  • Establish baseline CPU times for your rules under typical conditions
  • Understand that complex rules naturally consume more CPU
  • Factor in data volume - processing 1000 items will use more CPU than processing 10