Emergency Rental Assistance Program Improvement Act of 2021 or the ERAP Improvement Act of 2021 This bill modifies the Department of Treasury's Emergency Rental Assistance program to increase the portion of funding that state and local government grant recipients may use for administrative costs. The bill also encourages grant recipients to utilize these administrative funds to provide application assistance in coordination with public entities such as libraries, elementary schools, courts, and public housing. The program continues to provide grants to state and local governments to assist eligible households in paying rent, utility costs, and other housing costs incurred due to the COVID-19 pandemic.