I can't close a library that I'm not using anymore

The library is likely disabled. To enable a library, in the Project Explorer, right-click the library, and select Enable Library.