SEVERANCE AGREEMENT

Exhibit 10.1

THE BRYN MAWR TRUST COMPANY

EXECUTIVE CHANGE-OF-CONTROL

SEVERANCE AGREEMENT

This Agreement made as of April 4, 2005 between The Bryn Mawr Trust Company, a Pennsylvania financial institution, subject to the provisions of the Pennsylvania Banking Code of 1965, as amended (the “Company”), and James Duncan Smith (the “Employee”).

WHEREAS, the Employee is presently employed by the Company as its Executive Vice President, Treasurer and Chief Financial Officer;

WHEREAS, the Company considers it essential to foster the employment of well qualified key management personnel, and, in this regard, the board of directors of the Company recognizes that, as is the case with many financial institutions, the possibility of a change of control of the Company’s publicly held parent company, Bryn Mawr Bank Corporation, (“BMBC”) may exist and that such possibility, and the uncertainty and questions which it may raise among the Company’s management, may result in the departure or distraction of key management personnel to the detriment of the Company and ultimately to the detriment of BMBC and its shareholders;

WHEREAS, the Boards of directors of the Company and BMBC have determined that appropriate steps should be taken to reinforce and encourage the continued attention and dedication of key members of the Company’s management to their assigned duties, without distraction in the face of potentially disturbing circumstances arising from the possibility of a change of control of the BMBC, although no such change is now contemplated; and

WHEREAS, in order to induce the Employee, a key member of the Company’s management, to remain in the employ of the Company, the Company agrees that the Employee shall receive the

compensation and benefits set forth in this Agreement in the event his/her employment with the Company is terminated subsequent to a “Change of Control” (as defined in Section 1 hereof) of BMBC, as a cushion against the financial and career impact on the Employee of any such Change of Control;

NOW, THEREFORE, in consideration of the foregoing and the mutual covenants and agreements hereinafter set forth and intending to be legally bound hereby, the parties hereto agree as follows:

1. Definitions. For all purposes of this Agreement, the following terms shall have the meanings specified in this Section, unless the context clearly otherwise requires:

(a) “Affiliate” and “Associate” shall have the respective meanings ascribed to such terms in Rule 12b-2 of the General Rules and Regulations issued under the Securities Exchange Act of 1934, as amended (the “Exchange Act”).

(b) “AIP” shall mean any Annual Incentive Plan of the Company, as in effect immediately prior to a change of Control, or predecessor or prior plan, including BMBC’s Thrift and Savings Plan and the Company’s annual bonus plan.

(c) “Base Salary” shall mean the total cash remuneration earned by the Employee on an annualized basis in all capacities with the Company and its Subsidiaries, including, without limitation, any amounts the payment of which has been deferred by the Employee, excluding only payments earned by or allocated to the Employee under the AIP.

(d) A Person shall be deemed the “Beneficial Owner” of any securities:

(i) that such Person or any of such Person’s Affiliates or Associates, directly or indirectly, has the right to acquire (whether such right is exercisable immediately or only after the passage of time) pursuant to any

agreement, arrangement or understanding (whether or not in writing) or upon the exercise of conversion rights, exchange rights, rights, warrants or options, or otherwise; provided, however, that a Person shall not be deemed the “Beneficial Owner” of securities tendered pursuant to a tender or exchange offer made by such Person or any of such Person’s Affiliates or Associates until such tendered securities are accepted for payment, purchase or exchange;

(ii) that such Person or any of such Person’s Affiliates or Associates, directly or indirectly, has the right to vote or dispose of or has “beneficial ownership” of (as determined pursuant to Rule 13d-3 of the General Rules and Regulations issued under the Exchange Act), including without limitation pursuant to any agreement, arrangement or understanding, whether or not in writing; provided, however, that a Person shall not be deemed the “Beneficial Owner” of any security under this subsection (ii) as a result of an oral or written agreement, arrangement or understanding to vote such security if such agreement, arrangement or understanding (A) arises solely from a revocable proxy given in response to a public proxy or consent solicitation made pursuant to, and in accordance with, the applicable provisions of the General Rules and Regulations issued under the Exchange Act, and (B) is not then reportable by such Person on Schedule 13D under the Exchange Act (or any comparable or successor report); or

(iii) that are beneficially owned, directly or indirectly, by any other Person (or any Affiliate or Associate thereof) with which such Person (or any of such

Person’s Affiliates or Associates) has any agreement, arrangement or understanding (whether or not in writing) for the purpose of acquiring, holding, voting (except pursuant to a revocable proxy as described in the proviso to subsection (ii) above) or disposing of any voting securities of BMBC; provided, however, that nothing in this Section 1(d) shall cause a Person engaged in business as an underwriter of securities to be the “Beneficial Owner” of any securities acquired through such Person’s participation in good faith in a firm commitment underwriting until the expiration of forty (40) days after the date of such acquisition.

(e) “Board” shall mean the board of directors of the Company or BMBC as the context of this Agreement indicates.

(f) “Change of Control” shall be deemed to have taken place if (i) any Person (except BMBC, any Subsidiary of BMBC, any employee benefit plan of BMBC or the Company, any Person or entity organized, appointed or established by BMBC or any Subsidiary of BMBC for or pursuant to the terms of any such employee benefit plan) together with all Affiliates and Associates of such Person, shall become the Beneficial Owner in the aggregate of 25% or more of the common stock of BMBC then outstanding, or (ii) during any twenty-four month period, individuals who at the beginning of such period constituted the Board of BMBC or the Company cease, for any reason, to constitute a majority thereof, unless the election, or the nomination for election by BMBC’s or the Company’s shareholders, as the case may be, of each director who was not a director at the beginning of such period was approved by a vote of at least two-thirds of the directors in office at the time of such election or nomination, who were directors at the beginning of such period.

(g) “Common Stock” shall mean the outstanding common stock of BMBC.

(h) “Pension Plan” shall mean the BMBC non-contributory pension plan and the amended pension plan which covers eligible employees of the Company.

(i) “Person” shall mean any individual, firm, corporation, partnership or other entity.

(j) “Stock Plan” shall mean (i) BMBC’s 1986 Stock Option and Stock Appreciation Rights Plan, as amended and restated; (ii) BMBC’s 1998 Stock Option Plan; and (iii) any other stock option plan, stock option and stock appreciation rights plan, stock bonus plan, stock grant plan, or similar benefit plan established by BMBC and which exists for the benefit of the Employee at the time of a Change in Control.

(k) “Subsidiary” shall have the meaning ascribed to such term in Rule 12b-2 of the General Rules and Regulations issued under the Exchange Act.

(l) “Termination Date” shall mean the date of receipt of the Notice of Termination described in Section 2 hereof or any later date specified therein, as the case may be.

(m) “Termination of Employment” shall mean the termination of the Employee’s actual employment relationship with the Company.

(n) “Termination upon a Change of Control” shall mean a Termination of Employment upon or within two (2) years after a Change of Control either:

(i) initiated by the Company for any reason other than (x) the Employee’s continuous illness, injury or incapacity for a period of six consecutive months or (y) for “cause,” which shall mean misappropriation of funds, habitual insobriety, substance abuse, conviction of a crime involving moral turpitude, or gross negligence in the performance of his/her duties, which gross negligence has had a material adverse effect on the business, operations, assets, properties or financial condition of the Company and its Subsidiaries or BMBC and its Subsidiaries taken as a whole; or

(ii) initiated by the Employee following one or more of the following occurrences:

(A) a significant reduction by the Company or BMBC (if the Employee is an officer of BMBC) of the authority, duties or responsibilities of the Employee immediately prior to the Change of Control;

(B) any removal of the Employee from his/her officer position with BMBC, the Company and its Subsidiaries held by him/her immediately prior to the Change of Control, except in connection with promotions to higher office;

(C) a reduction by the Company in the Employee’s Base Salary as in effect immediately prior to the Change of Control;

(D) revocation or any modification of the AIP or Stock Plan, or any action taken pursuant to the terms of either plan, which materially (x) reduces the opportunity to receive compensation under any or both of such plans of equivalent amounts received by the Employee during the three (3) fiscal years immediately preceding the Change of Control, subject to the right of the Boards of Directors of BMBC or the Company, as appropriate, to establish in a manner consistent with past practice, prior to the Change of Control, reasonable goals under the AIP or Stock Plan, (y) reduces the compensation payable to the

Employee under either or both of such plans but which does not effect comparable reductions in the compensation payable to the other participants in such plans, or (z) increases the compensation payable to other participants in either or both of such plans but which does not effect corresponding increases in the amount of compensation payable to the Employee;

(E) termination or modification of BMBC’s Pension Plan or Supplemental Employee Retirement Plan, in each case as such plans are in effect immediately prior to the Change of Control, which materially reduces (x) the retirement benefits provided by such plans, or (y) the funding thereof provided by the Pension Plan or by any trust established by BMBC to fund benefits provided by the Supplemental Employee Retirement Plan;

(F) a transfer of the Employee, without his/her express written consent, to a location which is outside the Greater Philadelphia area (or the general area in which his/her principal place of business immediately preceding the Change of Control may be located at such time, if other than Bryn Mawr, Pennsylvania), or which is otherwise an unreasonable commuting distance from the Employee’s principal residence at the date of the Change of Control;

(G) the Employee being required to undertake business travel to an extent substantially greater than the Employee’s business travel obligations immediately prior to the Change of Control; or

(H) any failure of the Company to comply with and satisfy Section 13 of this Agreement.

2. Notice of Termination. Any Termination upon a Change of Control shall be communicated by a Notice of Termination to the other party hereto given in accordance with Section 14 hereof. For purposes of this Agreement, a “Notice of Termination” means a written notice which (i) indicates the specific termination provision in this Agreement relied upon, (ii) briefly summarizes the facts and circumstances deemed to provide a basis for termination of the Employee’s employment under the provision so indicated, and (iii) if the termination date is other than the date of receipt of such notice, specifies the termination date (which date shall not be more than 15 days after the giving of such notice).

3. Severance Compensation upon Termination. Subject to the provisions of Section 10 hereof, in the event of the Employee’s Termination upon a Change of Control, the Company shall pay to the Employee, within fifteen (15) days after the Termination Date (or as soon as possible thereafter in the event that the procedures set forth in paragraph (b) of Section 11 hereof cannot be completed within fifteen (15) days) an amount in cash equal to three (3) times the sum of the Employee’s Base Salary in effect either immediately prior to the Termination of Employment or immediately prior to the Change of Control, whichever is higher.

4. Other Payments. Subject to the provisions of Section 10 hereof, in the event of the Employee’s Termination upon a Change of Control, the Company shall:

(a) pay to the Employee within fifteen (15) days after the Termination date:

(i) unless the Employee has exercised such options, an amount equal to the excess, if any, of the aggregate fair market value of the shares of BMBC’s Common Stock subject to all stock options outstanding and unexercised as of

the Termination Date, whether vested or unvested, granted to the Employee under the Stock Plan, over the aggregate exercise price of all such stock options. For purposes of this paragraph, fair market value shall mean the highest of (x) the closing price of BMBC’s Common Stock on the last business day the Common Stock was traded immediately preceding the Termination Date, if such Common Stock is publicly traded at such date, (y) i