Gcode upload with python and OpenAPI

cnc65

I am trying to write a Python program to control a Duet 3 mainboard 6HC via Ethernet. I am using the OpenAPI definition for standalone mode. Everything works well except for the file upload function.

When I run my test program, I get this error message:

File Path exists: True
Exception when calling DefaultApi->rr_upload_post: (0)
Reason: Cannot prepare a request message for provided arguments. Please check that your arguments match declared content type.

Does anyone have any idea what I am doing wrong?

Here is my test program:

#! /usr/bin/python3
# -*- coding: utf-8 -*-
"""
Duet file upload test
"""
from __future__ import print_function
import os
from datetime import datetime
import swagger_client
from swagger_client.rest import ApiException
from pprint import pprint

"""
create an instance of the API class and connect Duet to
fixed host address DUET_HOST
"""
configuration = swagger_client.Configuration()
configuration.host = "http://192.168.2.1"

# instance of the API class
api_instance = swagger_client.DefaultApi(swagger_client.ApiClient(configuration))
time = datetime.now()
try:
    # new connection and log in
    duet_response = api_instance.rr_connect_get(password="", time=time)

except ApiException as e:
    print("Exception when calling DefaultApi->rr_connect_get: %s\n" % e)

# Upload a file
file = "f:/temp/test.g"
print("File Path exists: ", os.path.exists(file))

try:
    duet_response = api_instance.rr_upload_post(file)
except ApiException as e:
    print("Exception when calling DefaultApi->rr_upload_post: %s\n" % e)

# Disconnect Duet
try:
    duet_response = api_instance.rr_disconnect_get()
    # pprint(duet_response)
except ApiException as e:
    print("Exception when calling DefaultApi->rr_disconnect_get: %s\n" % e)

T3P3Tony

@cnc65 which version of the firmware are you using? Are you using the same version of the api yaml file in swagger to generate the api code. e.g. 3.6 is here for stand alone mode:
https://github.com/Duet3D/RepRapFirmware/blob/3.6-dev/Developer-documentation/OpenAPI.yaml

cnc65

RepRapFirmware: 3.5.4
OpenAPI yaml: 3.6 and also tested with 3.4

T3P3Tony

@cnc65 I am not a python expert but I used swagger to generate a python api using 3.6.0.

Looking at that code for your error message i found it in "rest.py":

  def request(self, method, url, query_params=None, headers=None,
                body=None, post_params=None, _preload_content=True,
                _request_timeout=None):
        """Perform requests.

        :param method: http request method
        :param url: http request url
        :param query_params: query parameters in the url
        :param headers: http request headers
        :param body: request json body, for `application/json`
        :param post_params: request post parameters,
                            `application/x-www-form-urlencoded`
                            and `multipart/form-data`
        :param _preload_content: if False, the urllib3.HTTPResponse object will
                                 be returned without reading/decoding response
                                 data. Default is True.
        :param _request_timeout: timeout setting for this request. If one
                                 number provided, it will be total request
                                 timeout. It can also be a pair (tuple) of
                                 (connection, read) timeouts.
        """
        method = method.upper()
        assert method in ['GET', 'HEAD', 'DELETE', 'POST', 'PUT',
                          'PATCH', 'OPTIONS']

        if post_params and body:
            raise ValueError(
                "body parameter cannot be used with post_params parameter."
            )

        post_params = post_params or {}
        headers = headers or {}

        timeout = None
        if _request_timeout:
            if isinstance(_request_timeout, (int, ) if six.PY3 else (int, long)):  # noqa: E501,F821
                timeout = urllib3.Timeout(total=_request_timeout)
            elif (isinstance(_request_timeout, tuple) and
                  len(_request_timeout) == 2):
                timeout = urllib3.Timeout(
                    connect=_request_timeout[0], read=_request_timeout[1])

        if 'Content-Type' not in headers:
            headers['Content-Type'] = 'application/json'

        try:
            # For `POST`, `PUT`, `PATCH`, `OPTIONS`, `DELETE`
            if method in ['POST', 'PUT', 'PATCH', 'OPTIONS', 'DELETE']:
                if query_params:
                    url += '?' + urlencode(query_params)
                if re.search('json', headers['Content-Type'], re.IGNORECASE):
                    request_body = '{}'
                    if body is not None:
                        request_body = json.dumps(body)
                    r = self.pool_manager.request(
                        method, url,
                        body=request_body,
                        preload_content=_preload_content,
                        timeout=timeout,
                        headers=headers)
                elif headers['Content-Type'] == 'application/x-www-form-urlencoded':  # noqa: E501
                    r = self.pool_manager.request(
                        method, url,
                        fields=post_params,
                        encode_multipart=False,
                        preload_content=_preload_content,
                        timeout=timeout,
                        headers=headers)
                elif headers['Content-Type'] == 'multipart/form-data':
                    # must del headers['Content-Type'], or the correct
                    # Content-Type which generated by urllib3 will be
                    # overwritten.
                    del headers['Content-Type']
                    r = self.pool_manager.request(
                        method, url,
                        fields=post_params,
                        encode_multipart=True,
                        preload_content=_preload_content,
                        timeout=timeout,
                        headers=headers)
                # Pass a `string` parameter directly in the body to support
                # other content types than Json when `body` argument is
                # provided in serialized form
                elif isinstance(body, str):
                    request_body = body
                    r = self.pool_manager.request(
                        method, url,
                        body=request_body,
                        preload_content=_preload_content,
                        timeout=timeout,
                        headers=headers)
                else:
                    # Cannot generate the request from given parameters
                    msg = """Cannot prepare a request message for provided
                             arguments. Please check that your arguments match
                             declared content type."""
                    raise ApiException(status=0, reason=msg)
            # For `GET`, `HEAD`
.....

Does that shed any light on why its failing?

Things to try:

Use the CURL commands as a test in the documentation here;
https://github.com/Duet3D/RepRapFirmware/wiki/HTTP-requests#post-rr_upload
That proves the duet side of things

Have thieving on the same version (preferably 3.6, so for RRF use 3.6beta4)

cnc65

@T3P3Tony Thank you for your help.
What I have tried so far:

Update RRF to 3.6beta4
Test CURL commands:

F:\temp>curl http://192.168.2.1/rr_connect?password=
{"err":0,"sessionTimeout":8000,"boardType":"duet3mb6hc102","apiLevel":2,"sessionKey":0}
F:\temp>curl --data-binary @"test.g" "http://192.168.2.1/rr_upload?name=/gcodes/test.g"
{"err":0}
F:\temp>

No success with Python so far.

What I dont understand is that the Python function generated from OpenAPI has only one parameter "name". The parameters "time" and "crc32" are optional. But at least two variables are required: source(path_and_filename.gcode) and destination(name) like in CURL.

rr_upload_post

Example

from __future__ import print_function
import time
import swagger_client
from swagger_client.rest import ApiException
from pprint import pprint

# create an instance of the API class
api_instance = swagger_client.DefaultApi()
name = 'name_example' # str | Path to the file to upload
time = '2013-10-20T19:20:30+01:00' # datetime | ISO8601-like represenation of the time the file was last modified (optional)
crc32 = 'crc32_example' # str | CRC32 checksum of the file content as hex string *without* leading `0x`. Usage of this parameter is encouraged (optional)

try:
    # Upload a file 
    api_response = api_instance.rr_upload_post(name, time=time, crc32=crc32)
    pprint(api_response)
except ApiException as e:
    print("Exception when calling DefaultApi->rr_upload_post: %s\n" % e)

Parameters

Name	Type	Description	Notes
name	str	Path to the file to upload
time	datetime	ISO8601-like represenation of the time the file was last modified	[optional]
crc32	str	CRC32 checksum of the file content as hex string without leading `0x`. Usage of this parameter is encouraged	[optional]

T3P3Tony

@cnc65 said in Gcode upload with python and OpenAPI:

so curl is working, there is an issue with the python implementation or the API description in YAML file

What I don't understand is that the Python function generated from OpenAPI has only one parameter "name". The parameters "time" and "crc32" are optional. But at least two variables are required: source(path_and_filename.gcode) and destination(name) like in CURL.

I think the "name" parameter is the destination path & filename. however the comments in the YAML file imply its the source:

parameters:
               - name: 'name'
                 in: query
                 description: 'Path to the file to upload'
                 required: true
                 schema:
                     type: string
               - name: 'time'
                 in: query
                 description: 'ISO8601-like represenation of the time the file was last modified'
                 required: false
                 schema:
                     type: string
                     format: 'date-time'
               - name: 'crc32'
                 in: query
                 description: 'CRC32 checksum of the file content as hex string *without* leading `0x`. Usage of this parameter is encouraged'
                 required: false
                 schema:
                     type: string

@chrishamm can you help?

cnc65

Is a solution to this problem in progress?

chrishamm

@cnc65 @T3P3Tony is right. name specifies the target filename (e.g. /gcodes/foobar.gcode) and the body must contain the file content. I don't know why the generated API does not provide an extra parameter for the file content, though - it's there in the OpenAPI definition.

T3P3Tony

@chrishamm can you point to where it is in the API definition? I.e. where is "body" is specified in the definition?

chrishamm

@T3P3Tony Here: https://github.com/Duet3D/RepRapFirmware/blob/3.6-dev/Developer-documentation/OpenAPI.yaml#L174 and here: https://github.com/Duet3D/RepRapFirmware/wiki/HTTP-requests#post-rr_upload

T3P3Tony

@chrishamm thanks so this is some issue with swagger?

chrishamm

@T3P3Tony Perhaps it was the GET request, not the POST request? I'm not sure.

T3P3Tony

@chrishamm I looked at the swagger post code.

cnc65

@chrishamm @T3P3Tony
I have changed the requestBody in the OpenAPI.yaml file so that a parameter body is generated in Python. This works for uploading gcode files. I have not tested other files.

requestBody:
	required: true
	description: 'File content'
		content:
			application/octet-stream:
			schema:
				type: string
				format: binary

The function call in Python then looks like this:

# Upload a file
file = "f:/temp/test.g"
name = "0:/gcodes/test-1.g"
try:
    body = open(file).read()
    duet_response = api_instance.rr_upload_post(body, name)
except ApiException as e:
    print("Exception when calling DefaultApi->rr_upload_post: %s\n" % e)