Swarthmore can be difficult to recruit to. Many D3 schools recruit students from all over the country. They are more National than a regional school. The Administration's at these schools, many times want the Diversity of the athletes to match the Diversity of the general population of the schools. This can require large coaching staffs and large recruiting budgets to see enough players who have the academic chops and are the proper level of athlete to play well at the D3 level. Then you have to convince the student to come to your schools over another that may have similar Academics, however has not performed as well on the baseball field.
If you look at Swarthmore's coaching staff it was 3 coaches a few years ago. There is just not enough time in the recruiting year to see the players you need to that can get into your school. This year they are up to 6. Midkiff has done a good job there considering the challenges. I have not taken the time to look. However I wonder how there other sports are doing. If the other sports are performing well, it may also indicate that the Administration has seen the benefit of having a vibrant and competitive Athletic department. They may have spent more time building athletics.