320 F. 2d 929 - Bisbee-Baldwin Corporation v. E Tomlinson

The question for decision is whether payments the taxpayer received for the termination of mortgage servicing contracts are taxable as ordinary income or as capital gains.1

This Court has recently decided three other cases concerning the tax effects of the termination of similar contracts: Nelson Weaver Realty Co. v. Commissioner, 5 Cir.1962, 307 F.2d 897; United States v. Eidson, 5 Cir.1962, 310 F.2d 111; Commissioner v. Maurice L. Killian, 5 Cir.1963, 314 F.2d 852. In Weaver, Judge Cameron, for the majority of the Court, held: 'It cannot be doubted that the sum total of the ingredients of (the mortgage company's) longstanding relationship with * * * (its) clientele constitutes a property right which is the equivalent of goodwill * * * a capital asset * * * entitled to capital gain treatment' on its sale. Judge Rives dissented: The 'part alloca(ble) to the right to receive service fees should be taxable as an ordinary income.'* In Eidson

* Footnote 6 of the majority opinion in Weaver points out that the question of allocation according to fragments was not presented or argued before the Tax Court or this Court.

In Eidson Judge Tuttle, for the Court, held that it 'makes no difference whether the assignment (of rights under the contract) * * * be denominated a sale'; the taxpayers assigned the right to receive 'net profit of operations during the remainder of the life of the contract', which must be treated as ordinary income. Killian followed Weaver 'to the extent that the facts * * * established a sale of good will', but the property sold consisted of 'expirations' not a bundle of rights as in Weaver, Eidson, and the case now before the Court.2

A majority of this Court conclude that they cannot bridge the gulf between Weaver and Eidson. Since we must choose between the two, we choose to follow Eidson on principle and the solution to the problem suggested by the dissent in Weaver.

The facts are not in dispute.

Bisbee-Baldwin, the taxpayer, is in the mortgage banking business. Most of its loans are secured by mortgages on residential property in Jacksonville, Florida. After making a loan, the company invariably assigns the mortgage to an institutional investor. The essential profit-making element is the investor's agreement to employ the mortgage company as its agent to service the mortgages. The company receives no profit on the assignment of a mortgage but earns an annual commission of one-half of one per cent of the principal outstanding balance of the mortgages serviced. The servicing activities generate other business. For example, the company often writes fire insurance on the property mortgaged, acts as real estate broker when the property is sold, and serves as property manager when a mortgage is foreclosed. Escrow deposits by the mortgagors enhance its credit standing, a substantial benefit since the company must borrow large sums from the banks in the operation of its affairs. Thus the success of the mortgage servicing business depends upon the amount of mortgage indebtedness it services.

The taxpayer had no right to assign the servicing agreement and could not demand any payment from a successor servicing agent if the investor transferred its business to another company. The taxpayer was not the exclusive agent for any investor, even in the Jacksonville area serviced by it. Each investor had the right to enter into similar agreements with other servicing agents, and the taxpayer had the right to assign and service mortgages for other investors.

During the fiscal year ending April 30, 1957, various investors cancelled servicing agreements with the taxpayer, and gave the business to other agents. When an investor cancels such an agreement without cause, it is customary for the investor to pay a termination fee equal to one per cent of the principal balance of the mortgages then being serviced by the mortgage company. In this case, several of the taxpayer's agreements with investors expressly provided for such a termination fee. The taxpayer received net termination fees of $206,454.63. The investors paid this sum to Bisbee-Baldwin, but were reimbursed by the new servicing agents for the amount of the termination fees paid to the taxpayer. In substance, therefore, the mortgage servicing was transferred from Bisbee-Baldwin to other agents for, as the district court found, the cancellations would not have taken place had the successor mortgage servicing agents not agreed to reimburse the investors in the amount of the termination fees.

'Plaintiff kept extensive files containing information concerning the mortgages which it serviced. When an investor canceled an agreement the files containing information concerning the mortgages assigned to that investor were turned over either to the investor or to the new servicing agent designated by the investor. Those files were of value to the Plaintiff in obtaining the (indirect) advantages (of servicing mortgages).'

Bisbee-Baldwin reported the termination fees as a net long-term capital gain of $206,454.43. The Commissioner ruled that the sum was ordinary income and assessed a deficiency of $45,705.35 against Bisbee-Baldwin. The taxpayer paid the deficiency and then brought suit for refund. The district court, citing the 'Weaver case (as) a situation very similar * * * agree(d) with the conclusion of the Tax Court that a mortgaging servicing contract is not a capital asset'. The district court also held that the 'termination of the mortgage servicing contracts by the various investors did not constitute sales or exchanges by the plaintiff.' Bisbee-Baldwin appeals from the judgment of the district court.

The fact that contractual rights in a mortgage servicing agency constitute a species of 'property' under state law affords no assistance in determining whether such rights are capital assets.3 As the Supreme Court stated in Commissioner v. Gillette Motor Transport Co., 1960, 364 U.S. 130, 134-135, 80 S.Ct. 1497, 1500, 1501, 4 L.Ed.2d 1617:

'While a capital asset is defined in 117(a)(1) as 'property held by the taxpayer,' it is evident that not everything which can be called property in the ordinary sense and which is outside the statutory exclusions qualifies as a capital asset. This Court has long held that the term 'capital asset' is to be construed narrowly in accordance with the purpose of Congress to afford capital-gains treatment only in situations typically involving the realization of appreciation in value over a substantial period of time, and thus to ameliorate the hardship of taxation of the entire gain in one year. * * * Thus the Court has held that an unexpired lease, * * * corn futures, * * * and oil payment rights, * * * are not capital assets even though they are concededly 'property' interests in the ordinary sense.'

In Hort v. Commissioner, 1941, 313 U.S. 28, 61 S.Ct. 757, 85 L.Ed. 1168, the Court held that amounts received in cancellation of a lease were taxable as ordinary income to a lessor. The Court said:

'Simply because the lease was 'property' the amount received for its cancellation was not a return of capital, quite apart from the fact that 'property' and 'capital' are not necessarily synonymous in the Revenue Act of 1932 or in common usage. Where, as in this case, the disputed amount was essentially a substitute for rental payments which 22(a) expressly characterizes as gross income, it must be regarded as ordinary income, and it is immaterial that for some purposes the contract creating the right to such payments may be treated as 'property' or 'capital." (313 U.S. 31, 61 S.Ct. 758)

This statement agrees with comparable statements in other cases that the capital gains section is an exception from the usual requirements and must be narrowly construed. See Corn Products Co. v. Commissioner, 1955, 350 U.S. 46, 76 S.Ct. 20, 100 L.Ed. 29; C.I.R. v. P. G. Lake, 1958, 356 U.S. 260, 78 S.Ct. 691, 2 L.Ed.2d 743; Burnet v. Harmel, 1932, 287 U.S. 103, 53 S.Ct. 74, 77 L.Ed. 199.

The question is, what do the mortgage servicing rights under the contracts represent. If they represent the right to earn future income in the form of commissions for services rendered, then the sum received for the cancellation of the contracts and the transfer of rights is ordinary income.

In Roscoe v. Commissioner, 5 Cir.1954, 215 F.2d 478, a sum received by taxpayers for their real-estate corporation stock in excess of the amount received by other shareholders was held to be ordinary income representing the commission which the taxpayers would have been entitled to had the land represented by the stock been sold directly. The Tenth Circuit recently reached a similar result in Wiseman v. Halliburton Oil Well Cementing Co., 10 Cir.1962, 301 F.2d 654. There the taxpayer had agreed to the termination of an exclusive license to use and to grant sub-licenses for a patented process in exchange for a non-exclusive license and one-third of the royalties received from third party licenses. The Court held that in substance the taxpayer had received the present value of income which it would otherwise have received in the future through the sublicenses. In so holding the Tenth Circuit expressly disapproved and in effect overruled Jones v. Corbyn, 10 Cir. 1950, 186 F.2d 450, on which the taxpayer relies. In that case, the Court gave a broad interpretation to the terms 'property' and 'capital assets', and held that money received from the sale of an insurance agency represented capital gains. Indeed, that case has long been recognized as being 'in large measure a departure * * * from the general principle governing sale of the proceeds from future services.' 3B Mertens, Law of Federal Income Taxation 22.34 (Zimet ed.). See 30 Texas L.Rev. 374 (1952).

The line between contractual rights representing capital assets and those representing the right to receive future income is far from clear. Judge Friendly, for the Second Circuit, after an extremely able, thorough survey of all the relevant cases, reached the following conclusion:

'One common characteristice of the group held to come within the capital gain provision is that the taxpayer had either what might be called an 'estate' in (Golonsky, McCue, Metropolitan), or an 'encumbrance' on (Ray), or an option to acquire an interest in (Dorman), property which, if itself held, would be a capital asset. In all these cases the taxpayer had something more than an opportunity, afforded by contract, to obtain periodic receipts of income, by dealing with another (Starr, Leh, General Artists, Pittston), or by vendering services (Holt), or by virtue of ownership of a larger 'estate' (Hort, P. G. Lake).' Commissioner v. Ferrer, 2nd Cir.1962, 304 F.2d 125, 130-131.

In Judge Friendly's analysis, as in Judge Rives's analysis in Weaver, some components of the 'bundle' of contractual rights held by a taxpayer are capital assets while others represent a substitute for future income. Thus in Ferrer the taxpayer's 'lease' of a play and his power, incident to the lease, to prevent a disposition of motion picture, radio, and television rights until after a certain date were capital assets. Cf. Commissioner v. Ray, 5 Cir.1954, 210 F.2d 390, cert. den., 348 U.S. 829, 75 S.Ct. 53, 99 L.Ed. 654. However, that part of the taxpayer's compensation for his contractual rights representing his right to forty per cent of the proceeds from the motion picture was taxable as ordinary income. We agree with this analysis. As Judge Rives said in Weaver:

'It seems to me, however, that a substantial part of the $121,841.11 paid to the Mortgage Company was for the right to receive service fees for the remaining years of the outstanding loans. To that extent the Mortgage Company was simply converting future income into present income. Commissioner v. P. G. Lake, Inc., 1958, 356 U.S. 260, 267, 78 S.Ct. 691, 2 L.Ed.2d 743.

'The sale of 'all of the rights, title, obligations and benefits pertaining to the servicing contract,' like the sale of any going business, should, I think, be comminuted into its fragments and the 'purchase price' should be allocated among the various assets sold. Williams v. McGowan, 2d Cir., 1945, 152 F.2d 570, 572; C.I.R. v. Chatsworth Stations, Inc., 2d Cir., 1960, 282 F.2d 132, 135. I would agree that a part of the $121,841.11 represented the purchase price of capital assets.'

Applying these principles to the factual situation before us, we find that the basic rights Bisbee-Baldwin sold were the annual servicing commissions on the principal balance outstanding on the mortgages. IndeedThe termination fee of one per cent of the mortgages serviced by the taxpayer was equivalent to two years gross income in commissions and was, to our minds, a substitute for the income which would have been earned by Bisbee-Baldwin had the contracts not been transferred.

This holding is, of course, in direct conflict with Nelson Weaver Realty Co. v. Commissioner, 5 Cir.1962, 307 F.2d 897. In an almost identical situation Weaver held that compensation for the assignment of a mortgage-servicing agency should be treated as capital gain. In reaching its decision, the Court relied heavily upon the fact that the amount paid the taxpayer 'bore no convincing resemblance to the amount the purchaser would actually collect out of the service contract sold since the purchaser had to pay the cost of collection.' It is equally true in the present case that the net income from commissions over a two year period would be much smaller than the sum received by Bisbee-Baldwin. But, as this Court recently stated in Eidson, 'while, of course, the sum of $170,000 which (the taxpayers) received in the taxable year at issue was doubtless much larger than the income they received in any year for a current year's operation, this clearly resulted from the fact that they were assigning, * * * not a capital asset whose value had enhanced or accrued over several years, but the right to recive this net profit of operations during the remainder of the life of the contract.' Since, according to undisputed testimony, the primary value of these mortgage correspondent relationships lay in the right to receive annual percentage commissions, there can be little doubt that the bulk of the sum paid to Bisbee-Baldwin was compensation for the loss of such commissions. In such a situation, the statement by the Supreme Court in Commissioner v. P. G. Lake, Inc., 1958, 356 U.S. 260, 78 S.Ct. 691, 2 L.Ed.2d 1143, is applicable.

'The substance of what was assigned was the right to receive future income. The substance of what was received was the present value of income which the recipient would otherwise obtain in the future. In short, consideration was paid for the right to receive future income, not for an increase in the value of the income-producing property.'

The taxpayer seeks to distinguish between the right to receive future income and the right to earn future income. This distinction finds no support in the decisions. Indeed, in Roscoe v. Commissioner, 5 Cir.1954, 215 F.2d 478, the very fact that the taxpayer's ten per cent commissions on the sale of real estate had to be earned by the taxpayer's efforts caused the compensation for this right to be classified as ordinary income.

In Gordon v. Commissioner, 5 Cir. 1958, 262 F.2d 413 this Court held that a lump-sum settlement of payments to be earned under an employment contract were taxable as ordinary income. In General Artists Corp. v. Commissioner, 2 Cir.1953, 205 F.2d 360, cert. den'd 346 U.S. 866, 74 S.Ct. 105, 98 L.Ed. 376, the commissions were to be earned. In Commissioner v. Pittston, 2 Cir.1958, 252 F.2d 344, cert. den'd 357 U.S. 919, 78 S.Ct. 1360, 2 L.Ed.2d 1364, in Leh v. Commissioner, 9 Cir.1958, 260 F.2d 489, and in Mansfield Journal Co. v. Commissioner, 6 Cir.1960, 274 F.2d 284 the profits taxed as ordinary income were future net profits. In Ferrer the royalties were to be earned by production of a play.

Still, some parts of the 'bundle' of contractual rights transferred by Bisbee-Bladwin were capital assets. The mortgage correspondent relationships have value in addition to the rights to servicting commissions. It acts as a 'feeder' for related businesses, such as insurance and real estate, frequently engaged in by mortgage bankers. The monthly escrow deposits made by the mortgagors considerably enhance the servicing agent's credit standing. Moreover, as the dissenting opinion in Nelson Weaver recognized, there is a sale of 'good will'. The mortgage portfolio of the mortgage banker tends to increase each year as both the mortgagors and the investors look to the mortgaging servicing agent for further funds and further outlets for investment. The taxpayer's extensive files and equipment are in the nature of capital assets. (Most of these were retained by the taxpayer.) These items are closely related to the everyday business operations of the taxpayer. They are not so integrally related, however, as to be insusceptible of separate valuation.

We summarize. Essentially, the contract was a management contract for the employment of personal services. The consideration received for the right to earn future servicing commissions must be regarded as a substitute for such future ordinary income. This important part of the bundle of rights sold or exchanged can be separated from the other parts and should be taxed for what it is-- not for what it is not. As in Ferrer: a transaction calls for

'In such instances, where part of a transaction calls for one tax treatment and another for a different kind, allocation is demanded, Helvering v. Taylor, 293 U.S. 507, 55 S.Ct. 287, 79 L.Ed. 623 (1935); Ditmars v. C.I.R., 302 F.2d 481 (2 Cir.1962). If it be said that to remand for this purpose is asking the Tax Court to separate the inseparable, we answer that no one expects scientific exactness; that however roughly hewn the decision may be, the result is certain to be fairer than either extreme; and that similar tasks must be performed by the Tax Court in other areas, see Webster Investors, Inc. v. C.I.R., 291 F.2d 192 (2 Cir.1961); Meister v. C.I.R., 302 F.2d 54 (2 Cir.1962) (determination of portion of purchase price attributable to good-will).' C.I.R. v. Ferrer, 2 Cir.1962, 304 F.2d 125, 135.

The case should be remanded to allow the district 'Court to allocate the amount (of the purchase price) according to the realities of the transaction.' Nelson Weaver Realty Co. v. Commissioner (dissent).

Looking only at the form of the transaction, it might be said that the payments fail to qualify for capital gains treatment because there was no 'sale or exchange' as required in Section 1221. Thus, the Commissioner argues that on termination of the contracts, Bisbee-Baldwin's rights vanished; nothing survived to be transferred. Commissioner v. Starr Bros., 2 Cir.1953, 204 F.2d 673, 674, held that cancellation of an exclusive agency did not result in capital gain. Such release not only ended the promisor's previously existing duty but also destroyed the promisee's rights. They were not transferred to the promisor; they merely came to an end and vanished.

In General Artists Corp. v. Commissioner, 2 Cir.1953, 205 F.2d 360, 361, this rule was applied, though here the 'sale' was to a third person. There the court said: 'But we think the correct view is that there was a release to the obligor of a negative covenant in order to allow a new covenant to be made with the third party.' See also Roscoe v. Commissioner, 5 Cir.1954, 215 F.2d 478, 482, cited and followed those cases. See, also, Leh v. Commissioner, 9 Cir.1958, 260 F.2d 489, 493-495. The district court followed these cases in holding that the 'elements which normally are involved in a sale or exchange are lacking in this situation.'4

In the fact pattern this case presents, where the 'realities of the transaction' undeniably show a market value for the bundle of rights said to have vanished, we feel compelled to look through form to substance. In substance the transaction was a two-party transfer. The investors were conduits: Bisbee-Baldwin received the payments; the transferees paid through the investors. Something was transferred. What Bisbee-Baldwin transferred was a bundle of rights under its contracts to service certain mortgages. For a price, the transferees stepped into Bisbee-Baldwin's shoes. It is irrelevant that the investors' approval was required, that the consideration was fixed in the contract or by trade custom, and that instead of assignment the transfer was effected by termination of the old contracts and execution of new contracts. These circumstances do not extinguish the fact that Bisbee-Baldwin gave up mortgage servicing commissions and other profits for a consideration. See Commissioner v. Ray, 5 Cir.1954, 210 F.2d 390, cert. den'd 348 U.S. 829, 75 S.Ct. 53, 99 L.Ed. 654; Commissioner v. Goff, 3 Cir.1954, 212 F.2d 875, cert. den'd 348 U.S. 829, 75 S.Ct. 52, 99 L.Ed. 654; Commissioner v. Golonsky, 3 Cir. 1952, 200 F.2d 72, cert. den'd 345 U.S. 939, 73 S.Ct. 830, 97 L.Ed. 1366; Commissioner v. McCue Bros. & Drummond, Inc., 2 Cir.1954, 210 F.2d 752; and Ayrton Metal Co. v. Commissioner, 2 Cir. 1962, 299 F.2d 741.

We hold therefore that property rights passed by sale or exchange from the taxpayer to the successor agents. This holding is circumscribed by our decision that not all of the stricks making up the bundle of rights constituted capital assets. As in Ferrer, although the whole bundle was sold or exchanged, the parties set no separate price upon each stick.

We therefore affirm the judgment below in part, reverse it in part, and remand the case for the district court to determine (1) what part of Bisbee-Baldwin's compensation was a substitute for future earnings, and therefore taxable as ordinary income and (2) what part or parts, if any, were capital assets, sold or exchanged, entitled to capital gains treatment.

GRIFFIN B. BELL, Circuit Judge (dissenting).

I dissent. I would affirm the judgment of the District Court without reaching the capital asset question, either as presented here or in Nelson Weaver Realty Co. v. Commissioner, 5 Cir., 1962, 307 F.2d 897. The sale or exchange there was undoubted. Weaver sold his mortgage banking business. The sole question was whether what was sold constituted capital assets.

Here there is a preliminary or threshold question of whether there was a sale or exchange. In fact, Bisbee-Bladwin refused to sell. Four out of several lending institutions represented by Bisbee-Baldwin shortly thereafter terminated their relationship with Bisbee-Baldwin. Two paid a termination fee required by contract. Two paid a negotiated fee in an amount customary in the trade. Three of the lending institutions chose the mortgage banker to represent them who had attempted to purchase their accounts from Bisbee-Baldwin, while the other chose a third mortgage banker.

The mortgage bankers reimbursed the lending institutions for the termination fees paid Bisbee-Baldwin, but by no stretch of my imagination can it be said that Bisbee-Baldwin made a sale to either of the lending institutions, or to the mortgage bankers who ended up with the accounts. Bisbee-Baldwin was simply terminated as the agent for these institutions, and continued in the mortgage banking business representing its remaining accounts.

The District Court, as did this court in United States v. Eidson, 5 Cir., 1962, 310 F.2d 111, decided the case on two grounds: there was no sale or exchange, and the subject matter of the alleged sale or exchange was not a capital asset. Nelson Weaver was decided to the contrary by our court after the District Court decision, but it does not help the taxpayer on the question of sale or exchange. And this ground remains as a sound basis in law and in fact for the decision of the District Court. Cf. Commissioner v. Starr Bros., Inc., 2 Cir., 1953, 204 F.2d 673; General Artist Corp. v. Commissioner, 2 Cir., 1954, 205 F. 360, cert. den., 346 U.S. 866, 74 S.Ct. 105, 98 L.Ed. 376; and Leh v. Commissioner, 9 Cir., 1958, 260 F.2d 489. See also Roscoe v. Commissioner, 5 Cir., 1954, 215 F.2d 478; and that portion of Eidson, supra, having to do with the question of sale or exchange, and Mertens Law of Federal Income Taxation, Vol. 3B, 22.92.

The rule that we are to look through form to substance can hardly suffice to transform the cancellation of an agency, over the protest of the agent, into a sale or exchange. The agent's rights simply came to an end. The language of the District Court set out in footnote 4 of the majority opinion, supra, amply demonstrates that there was no sale or exchange:

'The contracts provided for termination without cause by either party. Plaintiff could not have sold its rights under the contracts unless the investor gave approval. Plaintiff did not assign its rights. Its contracts were terminated and the investors then entered into new contracts with other agents. * * * Plaintiff had no choice in selecting the new agent. The sums received by plaintiff were determined by the termination clauses in its contracts with the investors or by trade custom. There was no bargaining as to price between plaintiff and the new agents. The elements which normally are involved in a sale or exchange are lacking in this situation.'

Thus it is that the law, in my judgment, is duly expanded in this ares by the majority opinion. That this has been done is all the more regrettable in the light of the unnecessary intra-circuit conflict that the majority establishes on the question of what constitutes a capital asset. Cf. Nelson Weaver, supra.

In my view, the judgment of the District Court should be affirmed for the reason that there was no sale or exchange. Any question as to the correctness of the Nelson Weaver capital asset rule should be postponed until a case presenting the question reaches us.