Agency Accountability Act of 2024 This bill requires the Director of Small and Disadvantaged Business Utilization of each agency that does not meet specified criteria for contracting with small businesses to testify before the Senate Committee on Small Business and the House Committee on Small Business and Entrepreneurship. Specifically, if an agency receives below a letter grade of A on its annual scorecard for small business contracting or does not meet two or more of the government-wide goals for contracting with small businesses, the director must testify before such committees about the reasons why the agency did not meet these benchmarks.