Paging Morpheus API in Python
Using Morpheus' API is something that comes up very often, and there have been a few bad habits that I've noticed. I myself have been guilty of using large maximum values when hitting endpoints. eg.
I came up with the following code. It's a function than concatenates the results into one variable and returns it. I may write this into the pymorpheus module with a proper cursor one day.
from pymorpheus import MorpheusClient # Set morpheusUrl and morpheusToken before this mclient = MorpheusClient(morpheusUrl, token=morpheusToken) def getAllResults(mclient, apipath): # Get first page of results sorting by ID result = mclient.call("get", apipath, options=[("sort", "id")]) # If the total is within the max results, no need to paginate if result['meta']['total'] <= result['meta']['max']: results = result else: # API calls in Morpheus return a key with a list, this grabs that first key mainkey = list(result.keys()) # Set the default offset offset = result['meta']['max'] moreResults = True while moreResults is True: # Get next result set, adding the offset nextResult = mclient.call("get", apipath, options=[("sort", "id"), ("offset", str(offset))]) # Add the nextResult list to the original result result[mainkey] = result[mainkey] + nextResult[mainkey] # If the offset plus the latest result size equals the total returned items, then exit if nextResult['meta']['offset'] + nextResult['meta']['size'] >= result['meta']['total']: moreResults = False # Else increment the offset and run again else: offset = offset + nextResult['meta']['max'] return(result) allNetworks = getAllResults(mclient, "/networks")