Esp32-Cam how to automatically refresh picture?

DavidG_68kg_PLA

Hey Guys,

I managed to get a static picture with the default Arduino Esp32-cam sketch in DWC, but not able to constantly refresh the picture (automatically with the refreshing setting).

Any ideas what i could try?

Best regards

droftarts

@DavidG_68kg_PLA How is your ESP32-CAM setup? Did you follow a tutorial like this one? https://randomnerdtutorials.com/esp32-cam-video-streaming-web-server-camera-home-assistant/

This should stream a jpeg, which is what the Duet needs. If it is updating every 5 seconds (like in the original thread), what setting do you have in Settings > General > Webcam update interval (ms)? 5000 is 5 seconds. Set it to 40 (ms) for 25 fps, or 0 for as fast as the ESP32-CAM can supply them, though DWC may struggle if the jpeg is high resolution.

If you got this working, it would be interesting to know how, as these cameras look pretty interesting.

Ian

Exerqtor

@droftarts said in Esp32-Cam how to automatically refresh picture?:

@DavidG_68kg_PLA ~~How is your ESP32-CAM setup? Did you follow a tutorial like this one? https://randomnerdtutorials.com/esp32-cam-video-streaming-web-server-camera-home-assistant/~~

This should stream a jpeg, which is what the Duet needs. If it is updating every 5 seconds (like in the original thread), what setting do you have in Settings > General > Webcam update interval (ms)? 5000 is 5 seconds. Set it to 40 (ms) for 25 fps, or 0 for as fast as the ESP32-CAM can supply them, though DWC may struggle if the jpeg is high resolution.

If you got this working, it would be interesting to know how, as these cameras look pretty interesting.

Ian

I've done the initial setup/proof of consept on mine and it works quite good, I pulled everything down again to swap for a autofocus sensor, and then I never got around to doing it. It's quite high on the to-do list now so I hope i get to it soon.

Have you had issues setting them up @droftarts?

droftarts

@Exerqtor I don't have any of the cameras, but at under £10 each, I'm certainly thinking of getting a couple. How did you set yours up? I can add them to the camera wiki page if I have the details: https://docs.duet3d.com/en/User_manual/Reference/DWC_webcam

I updated the documentation for Raspberry Pi-connected cameras recently. I got 2 cameras working and showing up in DWC, and Motion supports up to 4, though you have to show the Motion webserver page in DWC, not each individual camera feed. Motion supports IP cameras as well, so that would be a good addition too.

Ian

Exerqtor

@droftarts
Absolutely, the one I did I flashed as a "steaming web server" on a standard ESP32-CAM board with a OV2640 camera, and powered it by the 5V psu.

I set it up through this write up.

I plan on modding the board to accept a autofocus OV5640 camera like he did here, to see if that works better for my 350mm bed, since I had a hard time finding a good focus point with the OV2640.

Hopefully the autofocus won't panic too too much with the print head flicking around 🤣

DavidG_68kg_PLA

Hey @droftarts , i followed this instruction and tried your recommendations. But it didn‘t worked.

DavidG_68kg_PLA

Hey @Exerqtor,
would you share your code and settings you made in DWC?

Thanks a bunch guys

droftarts

@DavidG_68kg_PLA said in Esp32-Cam how to automatically refresh picture?:

Hey @droftarts , i followed this instruction and tried your recommendations. But it didn‘t worked.

If you could post a bit more information about what you have actually tried, then maybe I can help. It's impossibly vague to just say 'it didn't worked [sic]'.

Ian

DavidG_68kg_PLA

@droftarts of course, i´m sorry.

The IP-adress entered in DWC i tried with and without the ending "/capture"
Refreshingrates i also tried several 0, 40, 250 (as recommended in DWC or higher), 2000 (standard).
But none of these settings worked.
If i join the Wifi broadcasted by the RaspberryPi with my PC and use my browser i can reach the Cam via its ip (http://10.3.141.100/capture) and get a Picture, but not reliable.

The code i used is pasted in the code box and attached as .txt

Thanks in advance!

  Rui Santos
  Complete project details at https://RandomNerdTutorials.com/esp32-cam-video-streaming-web-server-camera-home-assistant/
  
  IMPORTANT!!! 
   - Select Board "AI Thinker ESP32-CAM"
   - GPIO 0 must be connected to GND to upload a sketch
   - After connecting GPIO 0 to GND, press the ESP32-CAM on-board RESET button to put your board in flashing mode
  
  Permission is hereby granted, free of charge, to any person obtaining a copy
  of this software and associated documentation files.

  The above copyright notice and this permission notice shall be included in all
  copies or substantial portions of the Software.
*********/

#include "esp_camera.h"
#include <WiFi.h>
#include "esp_timer.h"
#include "img_converters.h"
#include "Arduino.h"
#include "fb_gfx.h"
#include "soc/soc.h" //disable brownout problems
#include "soc/rtc_cntl_reg.h"  //disable brownout problems
#include "esp_http_server.h"

//Replace with your network credentials
const char* ssid = "T3000SE-L67W2";
const char* password = "1234567890";

#define PART_BOUNDARY "123456789000000000000987654321"

// This project was tested with the AI Thinker Model, M5STACK PSRAM Model and M5STACK WITHOUT PSRAM
#define CAMERA_MODEL_AI_THINKER
//#define CAMERA_MODEL_M5STACK_PSRAM
//#define CAMERA_MODEL_M5STACK_WITHOUT_PSRAM

// Not tested with this model
//#define CAMERA_MODEL_WROVER_KIT

#if defined(CAMERA_MODEL_WROVER_KIT)
  #define PWDN_GPIO_NUM    -1
  #define RESET_GPIO_NUM   -1
  #define XCLK_GPIO_NUM    21
  #define SIOD_GPIO_NUM    26
  #define SIOC_GPIO_NUM    27
  
  #define Y9_GPIO_NUM      35
  #define Y8_GPIO_NUM      34
  #define Y7_GPIO_NUM      39
  #define Y6_GPIO_NUM      36
  #define Y5_GPIO_NUM      19
  #define Y4_GPIO_NUM      18
  #define Y3_GPIO_NUM       5
  #define Y2_GPIO_NUM       4
  #define VSYNC_GPIO_NUM   25
  #define HREF_GPIO_NUM    23
  #define PCLK_GPIO_NUM    22

#elif defined(CAMERA_MODEL_M5STACK_PSRAM)
  #define PWDN_GPIO_NUM     -1
  #define RESET_GPIO_NUM    15
  #define XCLK_GPIO_NUM     27
  #define SIOD_GPIO_NUM     25
  #define SIOC_GPIO_NUM     23
  
  #define Y9_GPIO_NUM       19
  #define Y8_GPIO_NUM       36
  #define Y7_GPIO_NUM       18
  #define Y6_GPIO_NUM       39
  #define Y5_GPIO_NUM        5
  #define Y4_GPIO_NUM       34
  #define Y3_GPIO_NUM       35
  #define Y2_GPIO_NUM       32
  #define VSYNC_GPIO_NUM    22
  #define HREF_GPIO_NUM     26
  #define PCLK_GPIO_NUM     21

#elif defined(CAMERA_MODEL_M5STACK_WITHOUT_PSRAM)
  #define PWDN_GPIO_NUM     -1
  #define RESET_GPIO_NUM    15
  #define XCLK_GPIO_NUM     27
  #define SIOD_GPIO_NUM     25
  #define SIOC_GPIO_NUM     23
  
  #define Y9_GPIO_NUM       19
  #define Y8_GPIO_NUM       36
  #define Y7_GPIO_NUM       18
  #define Y6_GPIO_NUM       39
  #define Y5_GPIO_NUM        5
  #define Y4_GPIO_NUM       34
  #define Y3_GPIO_NUM       35
  #define Y2_GPIO_NUM       17
  #define VSYNC_GPIO_NUM    22
  #define HREF_GPIO_NUM     26
  #define PCLK_GPIO_NUM     21

#elif defined(CAMERA_MODEL_AI_THINKER)
  #define PWDN_GPIO_NUM     32
  #define RESET_GPIO_NUM    -1
  #define XCLK_GPIO_NUM      0
  #define SIOD_GPIO_NUM     26
  #define SIOC_GPIO_NUM     27
  
  #define Y9_GPIO_NUM       35
  #define Y8_GPIO_NUM       34
  #define Y7_GPIO_NUM       39
  #define Y6_GPIO_NUM       36
  #define Y5_GPIO_NUM       21
  #define Y4_GPIO_NUM       19
  #define Y3_GPIO_NUM       18
  #define Y2_GPIO_NUM        5
  #define VSYNC_GPIO_NUM    25
  #define HREF_GPIO_NUM     23
  #define PCLK_GPIO_NUM     22
#else
  #error "Camera model not selected"
#endif

static const char* _STREAM_CONTENT_TYPE = "multipart/x-mixed-replace;boundary=" PART_BOUNDARY;
static const char* _STREAM_BOUNDARY = "\r\n--" PART_BOUNDARY "\r\n";
static const char* _STREAM_PART = "Content-Type: image/jpeg\r\nContent-Length: %u\r\n\r\n";

httpd_handle_t stream_httpd = NULL;

static esp_err_t stream_handler(httpd_req_t *req){
  camera_fb_t * fb = NULL;
  esp_err_t res = ESP_OK;
  size_t _jpg_buf_len = 0;
  uint8_t * _jpg_buf = NULL;
  char * part_buf[64];

  res = httpd_resp_set_type(req, _STREAM_CONTENT_TYPE);
  if(res != ESP_OK){
    return res;
  }

  while(true){
    fb = esp_camera_fb_get();
    if (!fb) {
      Serial.println("Camera capture failed");
      res = ESP_FAIL;
    } else {
      if(fb->width > 400){
        if(fb->format != PIXFORMAT_JPEG){
          bool jpeg_converted = frame2jpg(fb, 80, &_jpg_buf, &_jpg_buf_len);
          esp_camera_fb_return(fb);
          fb = NULL;
          if(!jpeg_converted){
            Serial.println("JPEG compression failed");
            res = ESP_FAIL;
          }
        } else {
          _jpg_buf_len = fb->len;
          _jpg_buf = fb->buf;
        }
      }
    }
    if(res == ESP_OK){
      size_t hlen = snprintf((char *)part_buf, 64, _STREAM_PART, _jpg_buf_len);
      res = httpd_resp_send_chunk(req, (const char *)part_buf, hlen);
    }
    if(res == ESP_OK){
      res = httpd_resp_send_chunk(req, (const char *)_jpg_buf, _jpg_buf_len);
    }
    if(res == ESP_OK){
      res = httpd_resp_send_chunk(req, _STREAM_BOUNDARY, strlen(_STREAM_BOUNDARY));
    }
    if(fb){
      esp_camera_fb_return(fb);
      fb = NULL;
      _jpg_buf = NULL;
    } else if(_jpg_buf){
      free(_jpg_buf);
      _jpg_buf = NULL;
    }
    if(res != ESP_OK){
      break;
    }
    //Serial.printf("MJPG: %uB\n",(uint32_t)(_jpg_buf_len));
  }
  return res;
}

void startCameraServer(){
  httpd_config_t config = HTTPD_DEFAULT_CONFIG();
  config.server_port = 80;

  httpd_uri_t index_uri = {
    .uri       = "/",
    .method    = HTTP_GET,
    .handler   = stream_handler,
    .user_ctx  = NULL
  };
  
  //Serial.printf("Starting web server on port: '%d'\n", config.server_port);
  if (httpd_start(&stream_httpd, &config) == ESP_OK) {
    httpd_register_uri_handler(stream_httpd, &index_uri);
  }
}

void setup() {
  WRITE_PERI_REG(RTC_CNTL_BROWN_OUT_REG, 0); //disable brownout detector
 
  Serial.begin(115200);
  Serial.setDebugOutput(false);
  
  camera_config_t config;
  config.ledc_channel = LEDC_CHANNEL_0;
  config.ledc_timer = LEDC_TIMER_0;
  config.pin_d0 = Y2_GPIO_NUM;
  config.pin_d1 = Y3_GPIO_NUM;
  config.pin_d2 = Y4_GPIO_NUM;
  config.pin_d3 = Y5_GPIO_NUM;
  config.pin_d4 = Y6_GPIO_NUM;
  config.pin_d5 = Y7_GPIO_NUM;
  config.pin_d6 = Y8_GPIO_NUM;
  config.pin_d7 = Y9_GPIO_NUM;
  config.pin_xclk = XCLK_GPIO_NUM;
  config.pin_pclk = PCLK_GPIO_NUM;
  config.pin_vsync = VSYNC_GPIO_NUM;
  config.pin_href = HREF_GPIO_NUM;
  config.pin_sscb_sda = SIOD_GPIO_NUM;
  config.pin_sscb_scl = SIOC_GPIO_NUM;
  config.pin_pwdn = PWDN_GPIO_NUM;
  config.pin_reset = RESET_GPIO_NUM;
  config.xclk_freq_hz = 20000000;
  config.pixel_format = PIXFORMAT_JPEG; 
  
  if(psramFound()){
    config.frame_size = FRAMESIZE_UXGA;
    config.jpeg_quality = 10;//10 standard
    config.fb_count = 2;//2
  } else {
    config.frame_size = FRAMESIZE_SVGA;
    config.jpeg_quality = 10;//12
    config.fb_count = 2;
  }
  
  // Camera init
  esp_err_t err = esp_camera_init(&config);
  if (err != ESP_OK) {
    Serial.printf("Camera init failed with error 0x%x", err);
    return;
  }
  // Wi-Fi connection
  WiFi.begin(ssid, password);
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("");
  Serial.println("WiFi connected");
  
  Serial.print("Camera Stream Ready! Go to: http://");
  Serial.print(WiFi.localIP());
  
  // Start streaming web server
  startCameraServer();
}

void loop() {
  delay(1);
}
code_text

RN-Webcamserver.txt

Exerqtor

I've managed to dig out the ESP32-CAM and programmer and messed about a bit.

Got the OV5640 AF working, and I've got a steady stream on (pun intended) DWC with a 40ms refreshrate. I had some issues getting somewhat smooth FPS without attaching the external antenna to the ESP32 chip.

I'm currently trying to tweak the image settings in the hopes of getting better image quality.

Exerqtor

@droftarts & @DavidG_68kg_PLA ~~Okey guys, got this thing going quite nicely now.~~

I also use Rui Santos sketch, but as you can see, but i've added some extra notes and code to be able to run a OV5640 AF (for those who want that), and I've tweaked the camera feed to what i feel work quite good in DWC for both desktop and handheld devices.

/*********
  Rui Santos
  Complete project details at https://RandomNerdTutorials.com/esp32-cam-video-streaming-web-server-camera-home-assistant/
  
  IMPORTANT!!! 
   - Select Board "AI Thinker ESP32-CAM"
   - GPIO 0 must be connected to GND to upload a sketch
   - After connecting GPIO 0 to GND, press the ESP32-CAM on-board RESET button to put your board in flashing mode
  
  Permission is hereby granted, free of charge, to any person obtaining a copy
  of this software and associated documentation files.

  The above copyright notice and this permission notice shall be included in all
  copies or substantial portions of the Software.

  *********

  Exerqtor
  I've modified this sketch so that it can be used with a OV5640 AF camera module, and it's been "optimized" to be used as web camera for RepRapFirmare & DuetWebControl
  
  IMPORTANT!!! 
   - To use an OV5650 AF camera module without HW modifying the ESP32-CAM you must make sure to buy a camera module made to be used with ESP32-CAM
     - You must also install the "OV5640 Auto Focus for ESP32 Camera" library by Eric Nam 
   - I also wanted to add a little hint when it comes to flashing the ESP32-CAM using a ESP32-CAM-MB module:
     - To connect GPIO 0 to GND you press and hold the on-board IO0 botton on the ESP32-CAM-MB

*********/

// To use a OV2640 camera module uncomment the following
//#include "esp_camera.h"

// To use a OV5640 AF camera module uncomment the following
#include <ESP32_OV5640_AF.h>

#include <WiFi.h>
#include "esp_timer.h"
#include "img_converters.h"
#include "Arduino.h"
#include "fb_gfx.h"
#include "soc/soc.h" //disable brownout problems
#include "soc/rtc_cntl_reg.h"  //disable brownout problems
#include "esp_http_server.h"

//Replace with your network credentials
const char* ssid = "**********";
const char* password = "**********";

#define PART_BOUNDARY "123456789000000000000987654321"

// This project is made to use AI Thinker camera
#define PWDN_GPIO_NUM     32
#define RESET_GPIO_NUM    -1
#define XCLK_GPIO_NUM      0
#define SIOD_GPIO_NUM     26
#define SIOC_GPIO_NUM     27
  
#define Y9_GPIO_NUM       35
#define Y8_GPIO_NUM       34
#define Y7_GPIO_NUM       39
#define Y6_GPIO_NUM       36
#define Y5_GPIO_NUM       21
#define Y4_GPIO_NUM       19
#define Y3_GPIO_NUM       18
#define Y2_GPIO_NUM        5
#define VSYNC_GPIO_NUM    25
#define HREF_GPIO_NUM     23
#define PCLK_GPIO_NUM     22


static const char* _STREAM_CONTENT_TYPE = "multipart/x-mixed-replace;boundary=" PART_BOUNDARY;
static const char* _STREAM_BOUNDARY = "\r\n--" PART_BOUNDARY "\r\n";
static const char* _STREAM_PART = "Content-Type: image/jpeg\r\nContent-Length: %u\r\n\r\n";

httpd_handle_t stream_httpd = NULL;

static esp_err_t stream_handler(httpd_req_t *req){
  camera_fb_t * fb = NULL;
  esp_err_t res = ESP_OK;
  size_t _jpg_buf_len = 0;
  uint8_t * _jpg_buf = NULL;
  char * part_buf[64];

  res = httpd_resp_set_type(req, _STREAM_CONTENT_TYPE);
  if(res != ESP_OK){
    return res;
  }

  while(true){
    fb = esp_camera_fb_get();
    if (!fb) {
      Serial.println("Camera capture failed");
      res = ESP_FAIL;
    } else {
      if(fb->width > 400){
        if(fb->format != PIXFORMAT_JPEG){
          bool jpeg_converted = frame2jpg(fb, 80, &_jpg_buf, &_jpg_buf_len);
          esp_camera_fb_return(fb);
          fb = NULL;
          if(!jpeg_converted){
            Serial.println("JPEG compression failed");
            res = ESP_FAIL;
          }
        } else {
          _jpg_buf_len = fb->len;
          _jpg_buf = fb->buf;
        }
      }
    }
    if(res == ESP_OK){
      size_t hlen = snprintf((char *)part_buf, 64, _STREAM_PART, _jpg_buf_len);
      res = httpd_resp_send_chunk(req, (const char *)part_buf, hlen);
    }
    if(res == ESP_OK){
      res = httpd_resp_send_chunk(req, (const char *)_jpg_buf, _jpg_buf_len);
    }
    if(res == ESP_OK){
      res = httpd_resp_send_chunk(req, _STREAM_BOUNDARY, strlen(_STREAM_BOUNDARY));
    }
    if(fb){
      esp_camera_fb_return(fb);
      fb = NULL;
      _jpg_buf = NULL;
    } else if(_jpg_buf){
      free(_jpg_buf);
      _jpg_buf = NULL;
    }
    if(res != ESP_OK){
      break;
    }
    //Serial.printf("MJPG: %uB\n",(uint32_t)(_jpg_buf_len));
  }
  return res;
}

void startCameraServer(){
  httpd_config_t config = HTTPD_DEFAULT_CONFIG();
  config.server_port = 80;

  httpd_uri_t index_uri = {
    .uri       = "/",
    .method    = HTTP_GET,
    .handler   = stream_handler,
    .user_ctx  = NULL
  };
  
  //Serial.printf("Starting web server on port: '%d'\n", config.server_port);
  if (httpd_start(&stream_httpd, &config) == ESP_OK) {
    httpd_register_uri_handler(stream_httpd, &index_uri);
  }
}

void setup() {
  WRITE_PERI_REG(RTC_CNTL_BROWN_OUT_REG, 0); //disable brownout detector
 
  Serial.begin(115200);
  Serial.setDebugOutput(false);
  
  camera_config_t config;
  config.ledc_channel = LEDC_CHANNEL_0;
  config.ledc_timer = LEDC_TIMER_0;
  config.pin_d0 = Y2_GPIO_NUM;
  config.pin_d1 = Y3_GPIO_NUM;
  config.pin_d2 = Y4_GPIO_NUM;
  config.pin_d3 = Y5_GPIO_NUM;
  config.pin_d4 = Y6_GPIO_NUM;
  config.pin_d5 = Y7_GPIO_NUM;
  config.pin_d6 = Y8_GPIO_NUM;
  config.pin_d7 = Y9_GPIO_NUM;
  config.pin_xclk = XCLK_GPIO_NUM;
  config.pin_pclk = PCLK_GPIO_NUM;
  config.pin_vsync = VSYNC_GPIO_NUM;
  config.pin_href = HREF_GPIO_NUM;
  config.pin_sscb_sda = SIOD_GPIO_NUM;
  config.pin_sscb_scl = SIOC_GPIO_NUM;
  config.pin_pwdn = PWDN_GPIO_NUM;
  config.pin_reset = RESET_GPIO_NUM;
  config.xclk_freq_hz = 20000000;
  config.pixel_format = PIXFORMAT_JPEG; 
  
  // Change camera feed resolution here:
  config.frame_size = FRAMESIZE_XGA;
  config.jpeg_quality = 5;
  config.fb_count = 2;

  
  // Camera init
  esp_err_t err = esp_camera_init(&config);
  if (err != ESP_OK) {
    Serial.printf("Camera init failed with error 0x%x", err);
    return;
  }
  // Wi-Fi connection
  WiFi.begin(ssid, password);
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("");
  Serial.println("WiFi connected");
  
  Serial.print("Camera Stream Ready! Go to: http://");
  Serial.print(WiFi.localIP());
  
  // Start streaming web server
  startCameraServer();
}

void loop() {
  delay(1);
}

Regarding DWC settings I've got this:

Enable Webcam: checked/enabled
Webcam URL: http://ip-adress/
Webcam update intercall (ms): 34 equates to 30FPS
Do not append extra HTTP qualifier when relading images: checked/enabled don't know if this one matters

EDIT:

I've tried the OV5640AF a little bit now, and it's NOT recommended!

First off it seems like the autofocus puts too much stress on the ESP32, and the camera module gets so hot it literally burns your fingers without active cooling.

Just switched back to the OV2640 now and trying to see if i can get that to work better, and the other issue is that we more or less "need" a multi client stream solution unless you ONLY check on your printer from one device.

I'll try to get some more work done on this this evening.

Exerqtor

I finally had some more time to mess with this!

I've put together and compiled a working Multiclient firmware that utilize FreeRTOS to que the stream(s) with ESP32 Arduino Core 2.0.14 and Espressif 2.0.6 drivers, which adds support for a couple new sensors and dev boards. And some other under the hood improvements like lower stream latency/lag (at least in my case).

The new Arduino Core brings the posibility to use the slightly more expensive but much more compact Seeed Studio XIAO ESP32S3 Sense (which i'm looking into trying)..

Anywho, with this firmware you can have up too 10 active connected clients on each ESP32-CAM module so it's more flexible to use in my opinion, since I like to check in on the printer with my smartphone every now and then.

As of now I've opted to run on SVGA / 800X600 (with an OV2640) resolution but a higher JPEG quality to make the stream reasonably quick without too much loss in image quality/size.

Still doing some more testing and polishing then i'm going to put it on GIT ️

DavidG_68kg_PLA

@Exerqtor thanks a bunch for sharing! I really appreciate it! Really looking forward to your project

Exerqtor

@DavidG_68kg_PLA No problem, I'll try to have it up this week/weekend ️