You can use Arduino libs via ESP-IDF. The IDF environment comes with a lot of libraries on its own but it is a development environment first and foremost. The design of IDF is very open and easy to understand throughout the stack. It is very easy to integrate external libraries via CMake as well.