Interesting ideas interspersed with nonsense - by nikhil bhatla

28 Days of Survival Food: Day 4

nikhil — 2010-07-28T11:40:25-08:00

OK, I've made it this far. It's been roughly 72 hours since I last ate. My body feels very weak, especially my lower arms (elbow down) and lower legs (knee down), which have been aching in a way that reminds me of growing pains. Becca and I went to see Salt last night, and I couldn't even prop my feet up on the chair because they kept falling asleep. I bet my blood pressure is very low right now - maybe I'll measure it with one of those automated machines on my way home.

Yesterday morning I weighed myself -> 150 lbs.
Yesterday before bed I weighed myself -> 147 lbs.

I normally vary my weight 3 to 5 pounds from waking to sleep time, but normally my weight increases by that amount, not decreases, which is what happened yesterday. We'll see how my weight fluctuates over the next 4 weeks of experiment.

It's worth mentioning that I've been walking 6 miles a day during the fast (and I will for the rest of the experiment), because I've been walking to the lab and back. Weekends I'll probably walk much less.

I also started getting a headache on Day 2, though it seems to have gotten weaker as the days progressed. I'm not sure if it's acually gotten weaker, or whether my mind has been distracted by the aching of my limbs. On survival school I never felt myself getting a headache, I suspect because the overall experience just left no attention for it.

I've been sleeping very well in our bed, though I haven't taken any showers yet. Maybe tonight.

Tonight I will break my fast with a banana and at least a quart of honey tea, which is exactly what we got on Day 4 of survival school (you can see it in the second to last picture on the page). I'll probably eat when I get home, some time around 6-7pm. My next meal after that will be tomorrow night's dinner of survival stew. Hopefully I'll stop feeling this weakness.

I think this might actually mimic survival school pretty well. On survival school I started to feel emotional on Day 4, missing Becca, my parents and brother to tears. I had a strikingly similar experience this morning. As I walked to school, I listened to an acoustic song about someone dying, and unexpectedly the tears welled up big-time. I guess I'm more vulnerable in this state, more sensitive and less in control of my emotional mind.

So far so good. I think this might actually work.
]]>Comments:
]]>Nicky: I am very impressed Nikhil, but I must say the Jewish mother in me would like to feed you and I am going to resist that when I see you next. The emotional experience is interesting to me. On a side note, Salt must have been an interesting choice of movies to watch when you are so weak, with all of the crazy action and all. I hope your weakness resides. I look forward to your posts~ With admiration~n
]]> Write Comment

X

nikhil — 2010-07-26T12:26:15-08:00

Post survival school, I had an experience that for lack of a better word I'll call "X". Words like "inner peace", "happiness", and "bliss" come to mind, but none of them actually captures the feeling that I had. I've tried and failed to explain it to people who have never had a similar feeling, so maybe it's like explaining color to a blind man. All I know is that I did not have one shred of anxiety in my body, that I was astonished by the great pleasure of banal foods, and that I no longer understood why, for all my life, I had sought happiness. I felt spiritual for the first time.

This was my experience on reentry to the modern world. 28 days of survival school's miserableness resensitized me to the ridiculous comforts our civilization provides, which our prehistoric ancestors worked so hard to achieve. I felt how I guessed a baby might feel at the moment of birth, stunned by the raw unfolding of fresh experience and by the ease with which life comes.

Unfortunately, this experience of X diminished over time. As the weeks wore on, anxiety slowly crept into my body, food became less flavorful, and I began to seek happiness anew. I think the kicker came when I had my first test in grad school, about 2 months after coming back from the wilderness. The test-taking stress jolted me back to my old, semi-anxious, food-unappreciative, happiness-seeking self.

But I had had it. For 2 months, I had become a state of being that I never imagined possible, that I didn't even know existed, one that I never predicted survival school would induce, one that made me want to go back.

But that would be ridiculous. Survival school is miserable, it's cold and wet, and it drove me to pray: for the sun's warmth, for protection from lightning, and because of food. So there's no way I was going to do that again. I had sworn as much.

But for the past 3 years since, I've thought about my experience of X almost every day. I remember back to that time, and try to re-kindle the experience from my memory of it. But I only recover small fractions of the true feeling, and even these too, have diminished over time.

So could I could recapture X some other way? Maybe I could just do a "mini survival school", not as extreme as BOSS but on my own with a friend? In an attempt to do just that, last month Tunlin and I set out on an 8-day trek into Yosemite, survival-style. That meant no tent and taking what we took on survival school: some clothes, a poncho, a knife, and a wool blanket. I thought that 8 days might be enough time to get a partial experience of X, without having to do the full 28 day treatment. Another difference is that we brought plenty of food, though the same food from survival school (mostly lentils and vegetables).

By day 4, we were done with it. Done with the cold, sleepless nights. Done with the uncomfortable packs on our backs. Given the choice to just hike back to the car, we didn't have the discipline to stay out there and endure the discomfort. Despite the early departure, we held out hope that we might have some experience of X when we got back. Tunlin had predicted at least 10% X, but I just got a brief glimmer (and I don't think he got much more other than a nice nicotine-hit after 4 days without). I could see X for a moment in the distance, but then it was gone. Food tasted just the same as before we'd left. Clearly 4 days was not enough.

While we were out there, we thought a lot about X. I wanted to figure out what aspects of survival school contributed to what aspects of X, so that I could recapure X piece-meal if I wanted. I ended up breaking it down into 4 parts:

Survival school's minimal, monotonous food contributed to appreciating the magnificent pleasure locked inside food in the modern world.

The physical labor of hiking 10-15 miles a day with a backpack contributed to appreciating not having to do that in the modern world.

The discomfort of sleeping on the ground and enduring storms contributed to appreciating my bed and a roof over my head.

The social isolation of not being able to contact anyone I knew contributed to new deep-felt love for Becca, my family and my old friends.

Given this analysis and a desire to re-experience X, I've decided to try a new post-survival school experiment. Items 2-4 are hard to mimic while living in Boston in my normal life, but item 1 seems quite feasible. So I've decided to eat approximately the same food and quantity that we ate on survival school, for 28 days. So what's on the menu? Absolutely nothing other than water for the first 3 days. Then 25 days of 16 ounces of oats + water for breakfast, raisins and peanuts as snacks during the day, and 16 ounces of survival stew for dinner, consisting of lentils, quinoa, potatoes, carrots, and onions. Also, small amounts of vegetable bullion, salt, and pepper for flavor. I plan to cook the food in large batches, and then heat them up in the microwave every day before eating. I may break it up with some amaranth and some lamb, both of which we also got on survival school. And I may also throw in an apple every 2 weeks (also like survival school).

Maybe by trying to mimic my survival diet I can recapture my post-survival school spiritual appreciation for food. I plan to blog about this experiment as the days progress, over the next 4 weeks.

I think it's worth a shot, and as of yesterday, it has begun.
]]> Write 1st Comment

P2P Lending + SVM = $$$?

nikhil — 2010-07-07T13:20:01-08:00

Since 2004, I've been intrigued by microfinance. It all started when I read Mohammed Yunus' book "Banker for the Poor". In it, Yunus describes how he was able to improve the lives of poor women in Bangladesh by providing them with small loans (as little as $50 or less). Not only did the lenders get a reliable return, the borrowers got the means to pull themselves out of poverty. I'd never been that interested in investing (for a variety of reasons), but this win-win approach really appealed to me. When I looked into investing my own money, I found that most services for lenders positioned the loan more as a temporary donation (with almost 0% interest) than as a competitive investment. The money would also be sent far away, which made me uncomfortable, so ultimately I never lent any money.

Peer-to-Peer (P2P) Lending
When Prosper.com started providing peer-to-peer (P2P) loans in the United States in 2005, my interest was piqued again. With their service, anyone in the US could go online and request to borrow up to $25,000. Lenders could then go to the site, look at the borrower's credit grade and other listing information, and decide to lend as little as $50 per loan. If the borrower got enough investors, their loan amount would be fully funded and they'd receive the loan (a supposed "wisdom of crowds" approach). Prosper was set up as an auction, where the borrower specified the highest interest rate they're willing to borrow at, and lenders specified the lowest interest rate they're willing to lend at. So the final interest rate ended up somewhere in between. The loan term was for 3 years, with a fixed monthly payment of principal and interest.

By cutting the bank out as the middleman, borrowers were able to get unsecured loans at interest rates lower than what credit cards or banks offer (from 8-20%). Likewise, lenders were able to get interest rates higher than bank CDs and perhaps even the stock market. Again, a win-win situation: you help people buy a car or get an engagement ring, and you get a good interest rate in return.

So in 2008, I started lending money on Prosper. At first it seemed like it was going really well, with my loans averaging about 11% interest. I only lent to AA or A grade borrowers, so I thought that it was a relatively low-risk investment. Fast-forward to July 2010: even after diversifying with more than 100 loans, 13% have defaulted, leaving my annualized return at a meager 0.8%. So much for that idea...

Support Vector Machines (SVM)
As the defaults began to accumulate in 2009, I took a class on machine learning at MIT. Machine learning is a sub-field of computer science that focuses on developing software that can learn. What does that mean exactly? As an example, let's say that I want a computer to label faces in a photograph with each person's name. One approach would be to hand-code heuristics that identify each face. For example, any face with light brown skin, black hair, and thick eyebrows could be labeled "Nikhil". Of course, this approach is tricky and fraught with peril. Not only does a person have to figure out the feature values that uniquely identify an individual, the computer has to know what "hair" and "eyebrows" are, what those features actually mean. The first step is really time-consuming, brittle and perhaps impossible as the number of different unique faces increases. And the second step is still an unsolved problem in computer vision. In this heuristic approach, the computer doesn't learn: it's just programmed to label faces based on a set of human-specified features.

Contrast this to a machine learning approach. Rather than having a human figure out what sets of features values go with each person's face, the computer would be "trained" in the same way that people are trained: by just being shown faces and the names attached to those faces. Then it's the computer's job to figure out what feature values go with the names. This type of learning is called "supervised" learning. After being shown a large number of face-name pairs, the program is "tested" by being presented with a face it's never seen before. The program's job is to decide, of the labeled faces it's already seen, which is most similar to the new face, and then label it with that person's name.

I'm not sure if this sounds easy or hard to you, but in fact it is quite tricky and doesn't work very well, especially with complex visual stimuli such as faces. If you think about it, a face can be photographed from many different angles and under various lighting conditions. One amazing ability of the human brain is that it instantly identifies faces, while even the most cutting-edge computer programs are much less accurate and much slower.

So what does all of this have to do with P2P lending? What if instead of having to label faces (a hard problem of choosing from a set of n choices), the computer just had to identify loan requests that were likely to default (a simpler problem of choosing from a set of only 2 choices, will default or won't)? Labeling loans likely to default is a yes/no type of question, so maybe machine learning techniques could work in this more limited space. I was so excited by this idea that I decided to actually test it.

But before we get to the data analysis, let me explain the machine learning technique I used, which is called a support vector machine (SVM). I don't fully understand the math behind the technique, but I do understand graphically how it's supposed to work. So I'm going to use graphs to explain the concept.

For simplicity, let's just assume that we know only 2 things about each loan: the loan amount (with a range of $50-$25,000) and the borrower's credit score (with a range of 550-800). With this information, we can plot 10 imaginary loans on a 2-dimensional graph, as shown in Figure 1. 5 of the loans have defaulted (shown in red) and the remaining 5 are current or paid in full (shown in blue).

Figure 1: 10 imaginary loans plotted with respect to credit score and loan amount

Now, what a support vector machine does is find the line that best separates the 2 populations of points, as shown in Figure 2:

Figure 2: A support vector machine (SVM) finds the line that best separates the 2 data sets

The perfect separating line would have all defaulted loans on one side and all non-defaulted loans on the other. In the example above, perfect separation is impossible for a linear function, though possible with a more complex function that can wiggle around. To keep things simple, though, we'll focus on a linear separating function, which ends up looking like a straight line on a graph.

An SVM is a bit magical because it'll find the line that best separates the two categories of points, by minimizing the categorization error over the training set. The "training set" is the set of categorized points that the SVM gets to learn from before it draws its line (the 10 points in the current example). In the graph above, the error rate on the training set is 10%, because 1 of the 10 loans is on the wrong side of the line and miscategorized as not-defaulting, even though it did default.

Performance on the training set only means so much, though. An SVM's true utility is revealed on loans not included in the training set, where it can predict whether they will default or not depending on which side of the line they fall on. 2 new loans, whose fate the SVM does not know, are depicted as green and purple x's below. The green x is a $14,000 loan request from a borrower with a credit score of 690, and the purple x is a $6,000 loan request from a borrower with a credit score of 640.

Figure 3: Two new loans (the x's) plotted with respect to the separating line. The SVM predicts that the green x will default, while the purple x won't.

A naive investor might think that the larger loan to the more credit-worthy borrower is a better bet. But the SVM, having learned from the outcomes of the last 10 loans, would suggest skipping that loan and instead lending to the lower credit-score borrower, because they're predicted not to default.

In addition to categorizing a novel point, the SVM can also tell us how confident it is that its classification is correct. Graphically, this amounts to how far the point is from the separating line. The SVM is less confident about points closer to the line (near the category boundary), while more confident about points farther from the line. This subtlety will prove critical in the analysis described below.

Hopefully this explanation of an SVM makes some sense. One beauty of the SVM is that it can find the best separating line across n-dimensions, not just the 2-dimensions from the example. Prosper makes available all of their historical loan data, which includes over 100 features per loan and whether or not the loan has defaulted. In my analysis, I use this feature set to train the SVM, and then use the remaining historical data to test the SVM's prediction accuracy.

Before we see how well it does, I want to mention some brief technical details. All of the following analysis was done in Matlab (R2008a) using libsvm-mat-2.89-1 (released April 2009). I simply normalized my data in Matlab and then fed it to libsvm, and it did all the hard work for me.

Using an SVM to Predict Defaults on Prosper.com
Remember, our goal with all of this mathematical trickery is to create a system that will give a return on investment that's better than we'd get without it. This amounts to the SVM being able to identify loans that will default better than I can identify ones that'll default.

At the time I did this analysis (April 2009), 4.65% of my Prosper loans fell into the default category (which includes loans that are more than 1 month late). For me to benefit from an SVM, it should reduce this default rate significantly (ideally by half or more).

I trained the SVM on Prosper loan data from November 11, 2005 to October 1, 2007, the day before my first loan. Training was done on 35% of the total, feature-complete data set (7,108 of all 20,204 loans - more details in Appendix 1). Initial results were promising. If I had followed the SVM's default predictions for the loans in my portfolio, my default rate would have dropped 14% (from 4.65% to 4.03%). While a small effect, this was an encouraging start.

After further thinking, I realized that instead of just using the SVM's prediction alone to decide what to invest in, I could act only on those predictions that the SVM had confidence in. If I only invested in loans that the SVM was more than 90% confident would not default, I could further reduce my default rate by 76% (from 4.65% to 1.12%). I would invest in much fewer loans, but earn a better return overall.

Curious about whether this result was specific to my portfolio or generalized further, I retested on the broader set of all loans available after the training period (a total of 13,096 loans). I call this the "invest-in-everything" strategy. If I had invested in all loans the SVM predicted would not default with more than 90% confidence, my default rate would drop a whopping 85% (from 15.28% to only 2.33%)! Clearly the SVM is providing a significant improvement in accuracy, part of which is redundant with that ill-defined loan selection process that operates in my head. This is sketched out in Figure 4:

-><4.7%26%2337%3B>[label="my\nportfolio"];<4.7%26%2337%3B>-><1.1%26%2337%3B>[label="SVM"];<15%26%2337%3B>-><2.3%26%2337%3B>[headlabel="SVM"labeldistance=5labelangle=14];}'>

Figure 4: Proportion of Prosper loans that default with subsequent filters. If all loans are purchased, the default rate is 15%. If only loans in my portfolio are included, the default rate is 4.7%. If only the loans in my portfolio approved with high confidence by the SVM are included, the default rate is 1.1%. If all loans approved with high confidence by the SVM are included, the default rate is 2.3%.

It would be interesting to re-run this analysis today and see if the SVM still returns a significant advantage, as more than a year's worth of new data has accumulated. However, I can no longer find a way to download loan performance data from Prosper. It looks like they're no longer releasing it, which is unfortunate.

Using an SVM to Predict Defaults on LendingClub.com
After completing my analysis of Prosper in April 2009, I wasn't able to use it to invest in new loans because Prosper had temporarily shut down. They were revising their legal agreements to comply with SEC regulations, and no new loans would be processed during that time. So I turned my attention to another P2P lending service called LendingClub.com.

Unlike Prosper's auction method, Lending Club sets interest rates using a proprietary analysis of the loan application, purposefully avoiding the supposed "wisdom of crowds". In retrospect, this is a much better strategy. Although Lending Club loans have lower interest rates, the default rates are substantially lower: from June 2007 to April 2009, 7.6% of all Lending Club loans were 1 or more months late, while 21% of all Prosper loan dollars were 1 or more months late. By rejecting over 90% of borrower applications, Lending Club has improved performance for lenders.

So once again I focused my trusty SVM on a new set of data, to see if it could improve results much like it had for Prosper. In this case, though, I didn't haven any investments with Lending Club, so I couldn't compare the SVM's results with my portfolio's results. So I was only able to compare the SVM approach with an "invest-in-everything" strategy.

If the SVM is trained on the oldest 35% of loans and tested on the newest 65% (similar to what was done for the Prosper analysis above), the SVM default rate is no different than the naive default rate (of about 2.97%, or 79 of 2,662 loans defaulting). If a confidence threshold of 0.9 is used, the default rate actually increases to 3.85% (3 of 78 loans default)! (More details can be found in Appendix 2)

Note that this result depends on the training set used. If I train on the oldest 60% of loans (up to October 16, 2008) and test on the newest 40%, I am able to lower my default rate from 0.82% to 0.45% by only lending when the SVM is more than 90% confident that the loan won't default.

Both outcomes are diagrammed in Figure 5:

-><0.5%26%2337%3B>[label="SVM+%232"];<3.0%26%2337%3B>-><3.9%26%2337%3B>[label="SVM+%231"];}'>

Figure 5: Proportion of Lending Club loans that default with 2 filters. SVM #1 refers to an SVM that is trained on the oldest 35% of data. This results in an increase in default rate. SVM #2 refers to an SVM that is trained on the oldest 60% of the data. This results in a decrease in default rate.

Overall, it appears as if the Lending Club SVM can either increase or decrease the default rate, depending perhaps on the time period examined or overall amount of training data used.

Conclusions
So what did I learn from all of this?
Prosper has more defaulting loans, probably because it doesn't screen borrowers as well as Lending Club does.

Investing on Prosper siginficantly benefits from using an SVM to make loan decisions, especially when a high confidence threshold (90%) is used.

It's unclear whether investing on Lending Club benefits from an SVM: depending on the conditions, it can either help or hurt.

The SVM is a useful computational tool for categorization, especially when there are only 2 categories to choose from.
So how has all of this analysis influenced my lending? I'm currently lending on Lending Club with new criteria I derived from my Prosper loans that defaulted. So far I have 0 defaults, but the loans haven't had much time to default (varying between one month to one year).

I have no intention of making new loans on Prosper even with the SVM, as my non-SVM return on investment has been abysmal, the company seems a bit shady (for reasons you can find while Googling), and Lending Club seems to be a less riskier alternative.

Perhaps in the future I'll setup a service that will use a Lending Club SVM to automatically suggest loans to purchase. However, I'm a bit hesitant to use a program whose learned criteria I don't yet understand. Though maybe I just learned the hard way what the SVM could have told me all along...

If you've gotten this far and still have questions or comments, don't hesitate to email me. My current email address is at the top of the page.

--
Notes, credits, and links:

My synopsis and comments on Yunus' "Banker for the Poor"

Prosper.com - the first P2P lending service in the US. I got basically no return on my investment here, so I would not recommend them.

LendingClub.com - another P2P lending service in the US. I've only started investing here, and so far my experience has been positive.

Matlab - data analysis environment used

libsvm - SVM library used in Matlab for all analyses

All figures were generated with Google Chart Tools

--

Appendix 1: Prosper Analysis Details
I trained the Prosper SVM with 112 features (see below). The training data spanned November 11, 2005 to October 1, 2007 (7,028 loans). I chose this start date because it's the first date data was available, and this end date because it's one day before my first loan. This data set included 4,931 examples of non-defaulted loans, and 2,177 examples of defaulted loans (after removing loans with incomplete feature sets).

The test data set was comprised of either (1) my portfolio of loans or (2) all loans made from October 2, 2007 to October 16, 2008 (the last day before Prosper shut down to comply with SEC regulations).

A loan was classified as not defaulted if its Loan.Status was Origination delayed, Current, Late (less than 30 days), Payoff in progress, or Paid. Conversely, a loan was classified as defaulted if its Loan.Status was Charge-off, 1 month late, 2 months late, 3 months late, 4+ months late, Defaulted (Delinquency), Defaulted (Bankruptcy), or Defaulted (Deceased). A loan was not labeled if its Loan.Status was Repurchased or Cancelled.

The following 112 features were used when training and testing the SVM:

Features related to the borrower herself:

CreditProfiles.AmountDelinquent
CreditProfiles.BankcardUtilization
CreditProfiles.CurrentCreditLines
CreditProfiles.CurrentDelinquencies
CreditProfiles.DelinquenciesLast7Years
CreditProfiles.Income
CreditProfiles.InquiriesLast6Months
CreditProfiles.LengthStatusMonths
CreditProfiles.OpenCreditLines
CreditProfiles.PublicRecordsLast10Years
CreditProfiles.PublicRecordsLast12Months
CreditProfiles.RevolvingCreditBalance
CreditProfiles.TotalCreditLines
CreditProfiles.FirstRecordedCreditLine

A borrower's credit grade. Only 1 of the following can be true, leaving the rest false:
CreditProfiles.AA
CreditProfiles.A
CreditProfiles.B
CreditProfiles.C
CreditProfiles.D
CreditProfiles.E
CreditProfiles.HR
CreditProfiles.NC

A borrower's employment status. Only 1 of the following can be true, leaving the rest false:
CreditProfiles.Full-time
CreditProfiles.Part-time
CreditProfiles.Self-employed
CreditProfiles.Retired
CreditProfiles.Not employed
CreditProfiles.Not available

A borrower's occupation. Only 1 of the following can be true, leaving the rest false:
CreditProfiles.Accountant/CPA
CreditProfiles.Administrative Assistant
CreditProfiles.Analyst
CreditProfiles.Architect
CreditProfiles.Attorney
CreditProfiles.Biologist
CreditProfiles.Bus Driver
CreditProfiles.Car Dealer
CreditProfiles.Chemist
CreditProfiles.Civil Service
CreditProfiles.Clergy
CreditProfiles.Clerical
CreditProfiles.Computer Programmer
CreditProfiles.Construction
CreditProfiles.Dentist
CreditProfiles.Doctor
CreditProfiles.Engineer - Chemical
CreditProfiles.Engineer - Electrical
CreditProfiles.Engineer - Mechanical
CreditProfiles.Executive
CreditProfiles.Fireman
CreditProfiles.Flight Attendant
CreditProfiles.Food Service
CreditProfiles.Food Service Management
CreditProfiles.Homemaker
CreditProfiles.Investor
CreditProfiles.Judge
CreditProfiles.Laborer
CreditProfiles.Landscaping
CreditProfiles.Medical Technician
CreditProfiles.Military Enlisted
CreditProfiles.Military Officer
CreditProfiles.Nurse (LPN)
CreditProfiles.Nurse (RN)
CreditProfiles.Nurse's Aide
CreditProfiles.Pharmacist
CreditProfiles.Pilot - Private/Commercial
CreditProfiles.Police Officer/Correction Officer
CreditProfiles.Postal Service
CreditProfiles.Principal
CreditProfiles.Professional
CreditProfiles.Professor
CreditProfiles.Psychologist
CreditProfiles.Realtor
CreditProfiles.Religious
CreditProfiles.Retail Management
CreditProfiles.Sales - Commission
CreditProfiles.Sales - Retail
CreditProfiles.Scientist
CreditProfiles.Skilled Labor
CreditProfiles.Social Worker
CreditProfiles.Student - College Freshman
CreditProfiles.Student - College Sophomore
CreditProfiles.Student - College Junior
CreditProfiles.Student - College Senior
CreditProfiles.Student - College Graduate Student
CreditProfiles.Student - Community College
CreditProfiles.Student - Technical School
CreditProfiles.Teacher
CreditProfiles.Teacher's Aide
CreditProfiles.Tradesman - Carpenter
CreditProfiles.Tradesman - Electrician
CreditProfiles.Tradesman - Mechanic
CreditProfiles.Tradesman - Plumber
CreditProfiles.Truck Driver
CreditProfiles.Waiter/Waitress
CreditProfiles.Other

Information that's supposed to be about the loan listing itself, not the borrower (though you can see some, like HasVerifiedBankAccount, is about the borrower):
Listing.AmountRequested
Listing.BankDraftFeeAnnualRate
Listing.BorrowerMaximumRate
Listing.LenderRate
Listing.DebtToIncomeRatio
Listing.FundingOption
Listing.GroupLeaderRewardRate
Listing.HasVerifiedBankAccount
Listing.IsBorrowerHomeowner

What the loan will be used for. Only 1 of the following can be true, leaving the rest false:
Listing.Not available
Listing.Debt consolidation
Listing.Home Improvement Loan
Listing.Business Loan
Listing.Personal Loan
Listing.Student Loan
Listing.Auto Loan
Listing.Other

The following 36 features were ignored and NOT included when training or testing the SVM:

CreditProfiles.CreationDate
CreditProfiles.DatePulled
Listing.AmountFunded
Listing.AmountRemaining
Listing.BidCount
Listing.BidMaximumRate
Listing.BorrowerCity
Listing.BorrowerState
Listing.CreationDate
Listing.CreditGrade (duplicate of credit grade associated with the CreditProfile)
Listing.Description
Listing.Duration
Listing.EndDate
Listing.Images
Listing.GroupKey
Listing.Key
Listing.BorrowerRate
Listing.ListingNumber
Listing.MemberKey
Listing.PercentFunded
Listing.Status
Listing.StartDate
Listing.Title
Listing.LoanTermInMonths
Loan.AgeInMonths
Loan.AmountBorrowed
Loan.BorrowerRate
Loan.CreationDate
Loan.CreditGrade
Loan.DebtToIncomeRatio
Loan.GroupKey
Loan.Key
Loan.LenderRate
Loan.ListingKey
Loan.OriginationDate
Loan.Term

Appendix 2: Lending Club Analysis Details
I trained the Lending Club SVM with 20 features (see below). The first training data set spanned June 14, 2007 (the earliest available) through March 22, 2008 (1,430 loans). The end date was chosen so that the training data comprised 35% of all of the available data (as was the case with the Prosper training set). I thought this would be the best way to make the methods comparable. This data set included 1,199 examples of non-defaulted loans, and 231 examples of defaulted loans (after removing loans with incomplete feature sets).

The first test data set was comprised of the remaining 65% of data, spanning March 23, 2008 to April 28, 2009 (2,662 loans).

A loans was categorized as not-defaulted if its Status was Issued, Current, In Grace Period, Late (16-30 days), or Fully Paid. A loan was categorized as defaulted if its Status was Late (31-120 days), Default, or Charged Off. A loan was not categorized or analyzed if its Status was Cancelled or Removed.

The following 20 features were used to train and test the SVM:

Req (loan amount requested)
IntRate
Credit
DebtToIncomeRatio
Home.RENT
Home.OWN
Home.MORTGAGE
Home.NONE (Home.ANY gets all zeros)
MoInc (monthly income)
FICO (credit score)
EarliestCredit
OpenCreditLines
TotalCreditLines
RevolvingCreditBalance
RevolvingLineUtilization
InquiriesInTheLast6Months
AccountsNowDelinquent
DelinquentAmount
DelinquenciesLast2Yrs
PublicRecordsOnFile

The following 17 features were left out and NOT used in the SVM:

Bor (amount borrowed, usually redundant with Req)
AppDate
AppExp
IssuedDate
Title
Description
MoPayt
RemainingPrincipal
PaymentsToDate
ScreenName
City
State
Education
Associations
Code
MonthsSinceLastDelinquency
MonthsSinceLastRecord

]]>Comments:
]]>sundar: yes i did get this far - great to see you continue to do interesting stuff, this atleast is much better than those frikking worms you write about:)

Tuomas Talola: Have to say, I've never heard of Support Vector Machines. What you've done, I know as regression analysis. Nothing wrong with regression analysis itself, but using it to predict future returns or defaults is little dubious. This has been the case in financial markets over and over again. However, the work you have done seems quite thorough, I appreciate it. I'd be interested in more detailed results.
]]> Write Comment

Video of real bacterial chemotaxis

nikhil — 2010-07-04T19:02:35-08:00

(This is the 4th in a 3-part series on the biology of bacterial chemotaxis. Consider it a video bonus. Parts 1, 2, and 3 are also available.)

I finally found a video of bacteria chemotaxing, and I wanted to share it with you (so I could finally shut up about bacteria by reaching some sort of closure on the topic).

In the video, bacteria are chemotaxing to an attractant present in the center of the screen. If you look closely, you can see 2 individual bacteria labeled in red, moving to the center, overshooting, and then coming back.

You can compare the real bacteria's behavior to the results of my chemotaxis simulator (with linear adaptation turned on), as shown below. I think they're a pretty good match, at least at steady-state.

Video of bacteria chemotaxis (from the Van Oudenaarden lab)

Where the bacteria hang out after 20,000 iterations of the simulator
]]> Write 1st Comment

Here's hoping the bacteria don't crawl off the screen

nikhil — 2010-04-12T17:13:04-08:00

(This is the third in a 3-part series on the biology of bacterial chemotaxis. The first post described some basic ideas about biological analysis, and the second post introduced bacteria and their chemotactic behavior. And don't forget the bonus video post.)

Alright, now that the context has been set, I can finally dive into my bacterial chemotaxis simulator, which is what motivated this whole series of posts to begin with (which has taken far too many months to publish).

Bacterial Chemotaxis Interactive Simulator

What is it?

This is an interactive simulator of bacterial chemotaxis that I built in October 2008. It's "interactive" because you can change a bunch of settings and see how the virtual bacteria change their behavior. On the left side is the bacterial playground, where the bacteria (small white rods) swim around. Red spots indicate there's a chemoattractant there (say, sugar), and the brighter the red, the higher the sugar concentration. One of the bacteria is colored yellow (it's hard to see when the simulation is moving fast, but it's there), and that bacterium's current tumble probability and sugar concentration are shown on the right as it moves through the environment.

Why did I build this?

I built this simulator because I wanted to get a better understanding of how bacterial chemotaxis might work. The simulator has a bunch of bacteria swimming around in a world of different sugar concentrations, and I wanted to see what it would take to get the bacteria to properly chemotax, to swim to the highest sugar concentration and spend the most time there. As I mentioned before, I really believe that a good test of whether we understand something is whether we can build a simulation of that very thing. So building this simulator was my quick test for whether we (as a scientific collective) understand how bacterial chemotaxis actually works. Of course, I'm not even close to being an expert on chemotaxis, but I figured I'd give it a quick 'n dirty shot. Plus, I like writing web-apps.

When I first started making this simulator, I had the impression that a bacterium moves to the highest sugar concentration simply by tumbling less if the current sugar levels are high, and tumbling more if the current sugar levels are low. As I'll show below, this simple strategy is insufficient for chemotaxis (at least in my simulator given the various things I've tried). What I've found instead is that it's critical for the bacterium to keep a memory of the recent sugar concentrations it's experienced. To chemotax, the bacterium must change its tumble probability not only if sugar levels are changing, but also if sugar levels are staying the same. If the sugar levels are relatively constant, it needs to return to a basal tumbling probability. If it doesn't, it will swim away from areas of high concentration. This may be obvious in retrospect, but I found it quite surprising and unexpected. Detailed results of experiments with the simulator are described in the "What have I learned" section below.

How does it work?

As mentioned before, bacteria can either move in "runs" or they can tumble. A run means that a bacterium just moves straight in the direction it was going, while a tumble means that it randomly spins around and changes the direction that it's moving in. In this simulation, each bacterium gets a turn to move. On each turn, it randomly chooses whether to move forward one square, or to randomly choose a new direction to move in and then moves one square in that direction. At the end of its turn the bacterium updates its tumble probability, depending on the sugar concentration at the new location (and perhaps its sugar history, as described below).

So let me run through an example of precisely how a turn works. Let's say it's a bacterium's turn to move, and it's current tumble probability is 0.9. This is a very high probability, and says that there's a 90% chance that the bacterium will tumble on its turn. The program then picks a (pseudo) random number uniformly distributed between 0 and 1, and if that number is less than the tumble probability of 0.9, it will tumble. If it's greater than 0.9, it will just move forward. Let's say it draws a 0.4. So now it's going to tumble. Now, it draws a second random number to decide which direction to tumble in (in this simulator a bacterium is limited to just moving in 4 directions - up, down, left, and right). A bacterium has an equal probability of tumbling into any of the 4 directions. So let's say it draws a 0.2, which means it will move left. It then moves one spot to the left.

Now here comes the interesting part. Now that the bacterium has moved, it's time to update its tumble probability for the next turn. So what seems like a reasonable, simple formula for doing this? Let's imagine that if the sugar concentration is low, that means the bacterium isn't moving in a good direction, so maybe it should change its direction. And if the sugar concentration is high, the bacterium is in a good place, and should just keep swimming forward. It could just stop swimming, but maybe there's even more sugar if it keeps going the way its been going. One way to implement this sort of algorithm is to just set the tumble probability to 1 minus the current sugar concentration (which is limited to values between 0 and 1). So if the bacterium was at a spot with a sugar concentration of 0.13, it would update its tumble probability to 1 - 0.13 = 0.87. Conversely, if the sugar concentration were high at 0.9, it would have a tumble probability of 0.1.

This is the default update formula, and is specified when the Adaptation field is set to "None". Before I talk too much about what the Adaptation field is, let me explain the other parts of the simulator (on the right side, from top to bottom):

Where the bacteria hang out: This is a location "heat map" of where the bacteria hang out, with black shading indicating that bacteria spend more time there than spots with whiter shades. This is the ultimate test of whether the bacteria are successfully chemotaxing. If the heat map is nearly uniformly grey, the bactera hang out everywhere equally. If the heat map is dark in the center when the bacteria are exploring a circle gradient, that means that they're successfully chemotaxing.
Where the sugar is: This is a visual representation of where the sugar is in the bacteria's environment. As I mentioned above, brighter red indicates higher sugar concentrations. The default is a circle gradient, with the highest concentration in the center of the circle.
A row describing the current sugar concentration and tumble probability of the yellow bacterium.
Pause/Play button: Pauses the simulation when it's running, and plays the simulation if it's paused.
Next frame button: Runs only one frame (turn) of the simulation, pausing immediately after that turn is finished. You can use this to step through the simulation to watch individual bacteria, since it updates a bit too fast otherwise.
Frames: How many frames (turns) the simulation has run so far.

These next settings actually affect whether the bacteria successfully chemotax. So this is the stuff you should fiddle with to see how the bacteria's behavior changes.

Adaptation: This sets whether the bacteria adapt to the sugar they experience, and if so, how quickly they adapt. What do I mean by adaptation? I mean that a bacterium will "get used to" the current sugar concentration and readjust its tumble probability to a "normal" level, even if the sugar concentration isn't changing. This is just like how your eyes readjust so you can see in a darkened room. There are 3 options for this setting:
1. None: The default setting, this specifies that the bacteria don't adapt at all, and the tumble probability is just updated to 1 minus the current sugar concentration on each turn (as described above).
2. Tumble returns to steady-state (linear): When set to this option, a bacterium's tumble probability will update to approach the value of the adaptated tumbling field, minus the difference in current versus previous sugar concentrations. How much it will approach is specified by the adaptation rate field. As an example, let's say that a bacterium's tumble probability is currently 0.6, it's adaptation rate is 1% (a probability of 0.01), the adapted tumbling probability is 50%, the current sugar concentration is 0.2, and the previous concentration was 0.3. So the new tumble probability will be (0.6 - 0.01) - (0.2 - 0.3) = 0.69. This formula may seem a bit complicated, but notice that it takes into account 2 important things: an adjustment to adapt back to a baseline probability, and the fact that the tumble probability increases if the sugar concentration decreases, and decreases if the sugar concentration increases. Now that I write this out, it seems sort of complicated, but hopefully it makes a little bit of sense.
3. No tumble up, tumble otherwise (2 state): This is an update algorithm inspired from a paper by the van Oudenaarden lab at MIT (see citation at the end). Basically, a bacterium's tumble probability is set to 0 if the current sugar concentration is higher than the average sugar concentration from the last 3 turns. If it's lower than the average from the last 3 turns, the tumble probability is set to the adapted tumbling probability. The idea here is the same: if sugar levels are increasing, never tumble, but if they are decreasing or staying the same, tumble like crazy. Note that if this adaptation algorithm is used, the min-max tumbling setting below is ignored.
Adapted tumbling: The tumble probability that is used in the adaptation algorithms described above. If adaptation is set to "None", this field has no effect. The default value is 50%.
Adaptation rate: The rate at which a bacterium's tumble probability relaxes to the adapted tumbling probability. This only affects the simulation if adaptation is set to "Tumble returns to steady-state (linear)". The default value is 1%, meaning that if the current sugar concentration equals the previous sugar concentration, the tumble probability will be updated to be 1% closer to the adapted tumbling probability. So if the current probability is 0.6, and the adapted probability is 0.5, the new probability will be 0.59. It will get 1% closer on each turn until it reaches 50%, assuming the sugar concentration remains unchanged.
Sugar gradient: This is the shape of the sugar gradient. The options are:

Left/right Circle Circle gradient (default)

Diamond gradient Corner gradient
Min-max tumbling: The first value specifies the minimum tumble probability for a bacterium, and the second value sets the maximum. Think of this as a floor and ceiling for a bacterium's tumble probability, regardless of how the tumble probability is updated (which you can set with the adaptation settings below).

In addition to the options above, there are some extra options if you click on the "More options" link:

Room size: This sets the size of the bacteria's environment, which must be square. The default value is 56 spaces square.
Bacteria number: This sets the number of bacteria to simulate. The default value is 20 bacteria.
Bacteria speed: This sets the update rate for the simulation. The default value is 1000 turns per second per bacteria. That means that, assuming your browser and CPU can handle it, the simulation will run 1000 turns for each bacteria in 1 second.
Hang-out updating: This sets how many turns have to pass before the "Where the bacteria hang out" heat map is updated. The default value is to update the heat map after 100 turns.
Auto-stop after: This tells the simulator when to automatically stop running the simulation. Without this setting the simulation would just run as long as your browser window is left open. I added this option because it gives you a common stopping point at which to compare simulation runs with different option values. The default value is to stop after 100,000 turns for each of the bacteria.

So that's a comprehensive list of the various options available in the chemotaxis simulator.

So what have I learned from this?

Basically, I learned that adaptation is critical for bacterial chemotaxis. Adaptation can be thought of simply as using a "memory" of a previous time to alter current behavior, by comparing then with now. In fancy lingo, I'd say that adaptation is just a "temporal comparator". By keeping a memory of recent sugar concentrations, the bacteria are able to chemotax to the peak of the sugar gradient. Without this memory, with just the current sugar concentration, the bacteria cannot chemotax (at least in this simulation under the conditions I tested).

Here are the heat map results showing where the bacteria hang out in 2 different worlds using 3 different adaptation algorithms. Each heat map is the result of a single simulation which includes: 20,000 frames of simulation, 20 bacteria, 56x56 world size, 10-90% min-max tumbling, 50% adapted tumbling, and 1% adaptation rate.

Sugar gradient:
No adaptation:
Linear adaptation:
2-state adaptation:

As you can see above, when there isn't any adaptation, the bacteria hang out all over the place, so the heat maps look like a formless grey blur. If you actually watch the bacteria moving around, you can see that at low concentrations, the bacteria just tumble like crazy and find it difficult to leave because they're always tumbling. But at high concentrations the bacteria just sail straight through, eventually ending up back in low concentration territory, where they begin to tumble like crazy again.

With linear adaptation, you can start to see the shape of the sugar gradient in the heat maps of where the bacteria hang out. This means that the bacteria are chemotaxing somewhat. Finally, with 2-state adaptation, you get the clearest picture of the sugar gradient from the heat maps, indicating that the bacteria are efficiently chemotaxing and staying in the areas of high sugar concentration.

Alright, that's pretty much it folks. I have various caveats listed below, but if you've made it this far, you may just want to play around with the simulator some more. Hopefully I haven't been too verbose, as I optimized this post for for completeness and clarity over pithiness.

Caveats and other notes

Bacteria only move in 4 directions, not all 8 (or more).
My simulated bacteria seem to tumble too much, compared with the video in the previous post.
The real bacteria from the video seem to have a curvature to their paths, which the simulated bacteria don't have (they're runs are in straight lines).
Remember, this is just a simulator. I don't mean to draw strong conclusions about biology based on this web-app; it's more meant to be a fun toy that can be played around with, and that may also inspire novel non-virtual experiments.
It's worth noting that many other people have already made bacterial chemotaxis simulators. Many of these have been published in the academic literature. Many of them are more complex and more accurate than what I've built. But what's unique about mine is that it is a webapp that you can run for yourself in a web browser. You can tweak variables and tinker with things and see how the behavior of the simulated bacteria change. So I think it's pretty nifty.
One significant limitation of my simulator is that it models bacteria at a behavioral level, calculating the tumble probability as a function of the changing sugar concentration. After making this simulator, I really wanted to model bacteria at a molecular level, simulating the protein-protein interactions that actually cause the changes in tumble probability. Comparing such a simulation to real bacteria would be an excellent way to measure how much we really understand about how bacterial chemotaxis works. However, it's been a year since I made this simulator, and realistically I'm not going to make the molecular simulator, so I decided to blog about what I had so far.

I hope you enjoyed this web-app, and maybe even learned a little bit!

--
References:
- Mittal..van Oudenaarden 2003. "Motility of Escherichia coli cells in clusters formed by chemotactic aggregation." PNAS. PDF]]>Comments:
]]>becca: Yay! Es un simulador muy bueno.
]]> Write Comment

Bacteria don't have to be icky

nikhil — 2010-02-07T10:30:37-08:00

(This is the second in a 3-post series on biology. The first post gave background on the various levels at which biologists analyze organisms. This post discusses bacteria and an interesting behavior that they have called chemotaxis. The last post will present a web app that's an interactive simulator of this behavior. And don't forget the bonus video post. Yes, I had originally thought this would all fit into 2 posts, but sometimes the chunkier the better.)

Alright, now on to the meat of things. I'm going to try to explain what I understand about bacterial chemotaxis as clearly as possible, so that it's accessible to anyone who's finished high school. I must prepare you, though, that it may get somewhat technical. Feel free to ask any questions in the comments, or to shoot me an email.

Bacteria are microscopic, single-cell organisms that are considered to be the most simple form of independent life. "Simplicity", as used here, just refers to the fact that a bacterium is small and so has fewer parts compared to any other independent organism. (As an aside, viruses are even simpler [fewer parts], but since they rely on other organisms to reproduce, they can't be counted in the "independent" category.) The species of bacteria that I'll be focusing on in this post is Escherichia coli. Yes, this very same species of bacteria lives in your gut and helps with your digestion, and some strains are known to make people nastily sick. The strains in which chemotaxis have been studied are non-infectious, though.

Artist's rendition of an individual E. coli. The capsule-shaped object is the single cell (~3 microns [millionths of a meter] in length), and the hair-like tendrils are the flagella, which a bacterium uses as propellers to move around.

So what exactly is chemotaxis? Chemotaxis is an organism's ability to move toward certain chemicals, such as sugar. A bacterium is able to move because it has flagella, which are wispy hairs that can spin like a propeller to propel a bacterium in different directions. When flagella spin in one direction (e.g. clockwise), the bacterium moves ahead, but when flagella spin in the other direction (e.g. counter-clockwise), the flagella get all tangled up, making the bacterium spin around like a bottle. Below is a video of E. coli swimming around (flagella are really tiny so you can't seem them at this resolution):

E. coli swimming around. Note that no chemotaxis is occuring in this video, as the bacteria just appear to be swimming in all different directions. (I wanted to show a video of actual chemotaxis, but I wasn't able to find one online.)

As you can see by watching the video, bacteria seem to move smoothly along curved lines, almost forming circles. Occassionally, they slow down, twitch a bit, and then start moving in a new direction. Sometimes they collide with the non-moving bacteria, which I'm assuming either got stuck to the glass slide or are dead.

What people have learned by analyzing bacterial motility (movement) is that they seem to move in a way that's called a "biased random walk". When there's no attractant (sugar) lying around, bacteria just wander, covering the space with no preference for any specific location. They move along their curved paths (called "runs"), and every once in a while, they spin around (called "tumbles") and start on in a new random direction. This is the "random walk" part of the "biased random walk".

The "bias" comes in when there is some sugar lying around, and bacteria want to get to it. If you think about it, there are many strategies you can use to get to something you want. Humans have the ability to see (unlike bacteria), so if there's some chocolate lying on the kitchen counter, we just find it with our eyes and home right in. If we were blind, we might just constantly reach our hands out, feeling around the counter until we touched the chocolate. In a way, bacteria are like blind people. They don't have hands, but they do have tiny receptors on their "skin" (cell membrane) that can bind and sense sugar. Whenever there's sugar around, the sugar changes the probability of whether a bacterium is going to tumble or not. One way to think about it is that as the amount of sugar around a bacterium increases, it tumbles less, because it doesn't want to screw up the fact that life's getting better. But when the sugar starts to decrease, a bacterium "realizes" that it'd rather be back there where there's more sugar, and it tumbles more. Note that they don't seem to be able to choose which new direction they will take - they just spin around randomly and then start swimming again. They keep doing this until the sugar concentration starts increasing again. This concentration-dependent tumble probability introduces a "bias" into a bacterium's normal "random walk", such that it now spends more time at higher sugar concentrations, exhibiting a place preference it didn't show before.

Now, because bacteria are relatively "simple" and genetically tractable (I may write up a separate post to explain what this means - it's too much to go into now), it was somewhat straightforward for researchers to dissect the behavior of chemotaxis into it's molecular parts. I've already mentioned 2 important molecules. First, there are the chemoreceptors that actually sense sugar, which are called methyl-accepting chemotaxis proteins (MCPs). And second, there are the proteins that the flagella are made out of, called flagellin. By a combination of genetic and biochemical analysis, researchers have been able to find the molecules that connect the sensory input with the motor output. I won't go into them in too much detail, but here's a diagram of the molecules involved (circa 2004):

Molecules involved in bacterial chemotaxis. You can see the sugar-sensors at the top (MCPs), and the flagella at the bottom (the curvy blue thing). In between you can see the proteins CheW, CheA, CheZ, CheY, and CheB, which are important for communicating the external sugar concentration to the flagella.

The bottom-line is this: Researchers have been able to bridge the extremes of biological analysis, connecting the big, obvious changes in behavior with the tiny, hard-to-see changes in molecules. Now that's what biology is all about! Why do we do the things that we do? Because our molecules do the things that they do!

So this is what I knew of bacterial chemotaxis way back in October 2008. But then I wondered if what we thought to be true was actually true. How do we know that the diagram above is right? How do we know that all these molecules are sufficient to create what we actually observe bacteria doing in the real world? In my opinion, the best test of understanding is creation - if you can build from scratch that which you've observed, then that can be damn good evidence that you truly understand what's happening. Though I'm no biochemist, I am an engineer, so I decided to make a computer simulation of what I understood to be happening during bacterial chemotaxis. Instead of diving to the lowest level and simulating the molecules involved, I started out at an intermediate level, just simulating tumble probability directly as function of sugar concentration. And as it turned out, I saw something that I didn't expect, something which surprised me and which I'll share in the next and final blog post in this series.

Below is a teaser of sorts, a static, low-res screenshot of the simulator. The real-deal to follow...

---
Credits and reference:
- Drawing of single E. coli found online at asymptotia.com/2008/08/11/ecoli-stories/
- Video of E. coli swimming around comes from a lab at the Rowland Institute at Harvard: www.rowland.harvard.edu/labs/bacteria/movies_ecoli.html
- Diagram of molecules involved in chemotaxis came from a paper: Wadhams & Armitage 2004, "Making sense of it all: Bacterial chemotaxis", Nature Reviews Molecular Cell Biology (PDF)
]]>Comments:
]]>Ruggero: I suppose that tumble probability increases with sugar concentration, so the bacteria spend more time in the area they like most.

nikhil: That's an interesting idea which I haven't tried, but the conventional wisdom is actually the opposite. Based on what I've read, it seems that real bacteria actually tumble less when they're moving up a sugar gradient, and tumble more when they're moving down a sugar gradient, though they do adapt and return to a baseline tumbling rate. It would be interesting to try your idea in the simulator (which I just posted) and see what happens. Unfortunately, there's no easy way to do this in the interface provided, so you'd have to change the javascript to try it out.
]]> Write Comment

A name change

nikhil — 2010-01-03T14:55:18-08:00

I was reading an essay about art and neuroscience out loud to Becca. The more I read, the more insightful it appeared it could have been, yet the more like computer-generated spam it actually sounded. Becca said "That person sounds like they have a lot of interesting ideas, interspersed with nonsense." For some reason, that made me think about my blog. So I decided to change its name, transforming it from the unassuming "Some thoughts and photos" to the whimsical "Interesting ideas interspersed with nonsense." If silence on a post's comment board indicates nonsense, and non-silence indicates interest, then quite a fitting name it is.
]]>Comments:
]]>omar: This is nonsense

david: this is silence. oh, wait.

Ruggero: This is a comment board.
]]> Write Comment

A Note about Biology

nikhil — 2009-12-25T03:11:56-08:00

(This is the first in a 3-post series on biology. I begin with some background, mostly to provide context for the second post, which is about a behavior in bacteria called chemotaxis. The third post will introduce a web app that simulates bacterial chemotaxis, and will be explained in gratuitous detail. I made it over a year ago, and it's about time that it saw the light of day. And yes Sundar, I know you don't find biology interesting, but maybe one day you'll agree with me that it's the most interesting thing in the world, far more interesting than business or technology. Also, don't forget the bonus video post.)

A biologist's task is to understand how those objects that we think of as "alive" work. Just as a curious engineer might ask, "How does a car work?", a biologist might ask, "How does a bacterium work?". But biology is actually very different from engineering. In the first case, an engineer deconstructs a human-made device, while in the second, a biologist dissects an organism of unknown origin following unknown rules. Biologists often revert to using engineering-style language (e.g. calling an organism a "system") as a linguistic shortcut, but this is more metaphor than actual truth. Biological organisms seem to be fundamentally different than devices that we've engineered so far, and part of the reason I find biology so intriguing is because I'm curious about the nature of this difference. Is it just that biological organisms are grounded in messy biochemistry, while most devices we create are built on far "cleaner" substrates? Or is there a basic biological principle that is currently missing in our engineered constructions?

In trying to understand how an organism works, biologists tend to work at 3 different levels. Ordered from the macroscopic to the microscopic, these levels are 1) the behavioral level, 2) the physiological level, and 3) the molecular level.

When biologists study behavior, they study the whole organism in its environment, including how it moves through the world, how it interacts with other organisms, how it eats, how it reproduces, how it poops, etc. They're looking for macroscopic events that can be observed just by looking at the organism usually without the aid of any additional tools (except, say, a microscope in the case of really small creatures). In the case of bacteria, the behavior under study could be the bacterium's ability to navigate through a gradient of sugar concentrations until it reaches the area of highest concentration. This behavior is called "chemotaxis".

When biologists study physiology, they're looking one level deeper into the organism. Take the example of bacterial chemotaxis. Rather than just observing that bacteria move to the peak of a sugar gradient, a physiologist will be interested in whether the electrical activity or calcium concentration within a bacterium changes as a function of where the bacterium is in the gradient. They'll be curious about whether this lower-level activity might actually be part of the physical mechanism that senses the sugar and determines how the bacterium moves through it. To start all the physiologist has is a correlation, but by mucking with the signal of interest (by, say, injecting electrical current), they can provide evidence for a causal role as well. Physiologists want to peer beyond behavioral observation into the mechanisms that might underlie a phenomenon. The physiological level is intermediate between the behavioral and molecular levels.

The lowest level that biologists tend to study is the molecular level. In the example of bacterial chemotaxis, the molecular biologist might try to remove molecules and see if the bacterium can still chemotax. For example, they might mutate the DNA of a bacterium, turning off different genes until they find a mutant bacterium that can no longer chemotax. Then they know that the normal functioning of that specific gene is required for chemotaxis, and have added a new molecule to the grab-bag of molecules that are believed to be important for chemotaxis.

The line between molecular biology and physiology can blur. In the example above, I mentioned calcium concentration as a signal that physiologists look at. But of course calcium is also a molecule, so in that case it can be thought of as a molecular analysis as well.

So who really cares about all this stuff? In my opinion, all levels matter if you want to answer the question of "how does a bacterium work?" What's funny is that in my experience, biologists tend to pigeon-hole themselves into looking at one level of analysis exclusively. For example, the researchers in my lab often focus on the molecular level, finding genes that may affect other molecular or cellular phenomenon (but not behavior). I always wonder whether that's really useful. I mean, who's to say that if gene A changes the concentration of molecule B, that it means anything to the functioning of the organism itself? It might just be the way it is, for no real rhyme or reason, with no real function or purpose. So while you've made a valid observation, I'm not sure why I should find it interesting...

For me, biology is all about behavior, or other macroscopic effects (such as disease and death) that really affect the survival of the organism. If you believe that evolution by natural selection contributed to how organisms came to be the way they are, then whatever it is that contributes to reproductive success is what really matters to the functioning of an organism. Whether molecule A is required for the survival of cell B may be of no consequence to the organism as a whole, so figuring out what molecule A is isn't really very interesting to me.

I am really curious about how an organism can do a behavior, and when I study behaviors I want to understand them all the way down to the molecular level.

Within biology, an organism's behavior is notoriously difficult to study, because it depends on so many variables; it's hard to know which ones matter and to control them all. When I first heard that bacterial chemotaxis was one of the most well-understood behaviors in biology, I was intrigued to understand what that meant. So far most biological explanations had left me wanting more, and here stood a so-called "crown jewel" of biological understanding. After reading a bunch of papers, I decided to take a stab at building a computer simulation of how some people thought that bacterial chemotaxis might actually work. By building a simulation and tweaking individual variables, I thought that I would gain a better understanding of how bacteria chemotax. I'll describe bacteria chemotaxis in more detail in the next post, and present my simulation as the grand finale.
]]>Comments:
]]>Ruggero: Excellent post. Biological systems are far more complex than any of the system we've been able to come up with so far. So far. There'll be a time when the products of our minds will be more complex and smarter than us all. And probably at that time it won't make much sense anymore to talk about 'products of our minds' and 'us' because they will merge one into the other. If you think more carefully, this actually already happened. Do you wear contact lenses? Do you have a tooth filled? Do you have a hip replacement? Do you know of Oscar Pistorius? Technology is rough, but it's slowly integrating with biology. And not only at the macroscopical level. The most interesting works are in nanotechnolgy, where the distinction between an engineer and a chemist (or a biologist) becomes pretty hard to make. Biological systems of course are still much smarter than robots. This is partly because biology writes information on DNA (about 3 nm), where current technology only goes down to 32 nm. And partly because we don't know how to handle matter properly yet. What is DNA if not a self-assembly material? 'Cleaner' substrates? There's nothing really clean even in engineering, believe me. 'Simpler' would sound much better. Is there a basic biological principle we are missing? I don't know. I only know that there's a lot more to know about the language used in nucleic acids. If the principle is there, it has to be written down somewhere. A question: does a virus chemotax?

nikhil: As far as I know, a virus doesn't have any ability to actively move through space. A bacterium, on the other hand, uses flagella as propellers to execute a biased random walk through space. So I guess that a single virus wouldn't be able to chemotax. I haven't really looked into it though, and perhaps the right experiment hasn't even been tried, so I could be wrong.

Sundar: Nice post, I do find biology interesting when written this way. I hated it because I did not want any part of dissection and infact, refused to in high school. PS - engineers do not always deconstruct human made devices, sometimes it is also nature like "Big bang" - unless you are differentiating between engineers and scientists

nikhil: yep, i'm distinguishing engineers, who build things, from scientists, who try to understand how non-human-made things (nature, socieities, human minds) might work. oh how vegetarian of you :)
]]> Write Comment

Cell phones, gestures, and 3G

nikhil — 2009-12-23T11:14:04-08:00

(I'm sitting on a plane as I write this, flying to LA for Christmas. It's a 6 hour flight, and since I seem to be slightly phone-obsessed at the moment, I figured I'd pass the time by putting my thoughts down, since there's nothing on TV and I'm bored. Be forewarned - my blog is not turning into yet another gratuitous tech review site, and the irregularly scheduled thoughts on science and life will be back shortly. But here's some tech talk, for all my techie friends :)

I've been quite impressed by the quality of the recent Android phones (esp. the HTC Eris on Verizon and the HTC Hero on Sprint - the Motorola Droid is an unceremonious brick). They seem to have finally closed the gap with the iPhone, in terms of being a full touch screen with a bunch of apps. They even look a lot like the iPhone. But the user interface still seems to have some shortcomings.

Anytime I use something, I've always been particularly sensitive to how I interact with a device, most often dictated by its user interface. This is an instinct - I don't think I ever learned to be this way, it's just how I came. While working at Google, I refined these sensitivities quite a bit. So the fact that Android's web browser and maps don't support pinch-zoom leaves me amiss. I suspect some sort of Apple-related patent-fear or side-deal on the part of Google, a fear that Palm seems to lack since the Palm Pre does multitouch zoom just fine (maybe Google doesn't have as many touch-screen patents as Palm does, so they have more patent-fear). Also, the Android user interface seems a bit more kludgy and disorganized than the iPhone - apps in multiple places, too many physical buttons, options that shouldn't be hidden in menus are hidden in menus, stuff like that.

After playing with these 3 phone UIs (iPhone, Android, and Palm's webOS), I must admit that my favorite is the one on Becca's Palm Pre (which runs webOS). We've both had Palm Treos for quite a few years (my 650 is going on 6 years old, and still works as well as the day I got it). I'm not a Palm fanboy, as there are many problems with my Treo, but I do think that Palm did a good job with the Pre. What makes the Pre UI better than these other UIs? The plentitude of intuitive (to me) gestures and the dedicated gesture area at the bottom of the screen.

Most people have probably never seen a Pre (or the cheaper Pixi), let alone used one. I didn't realize how much I relied on the Pre's gestures until I was using a new Android phone and unintentionally tried to go back one screen by using a Pre gesture, sliding my thumb from right to left at the bottom of the screen. The Pre's gestures make so much sense to my brain that, after only an hour or so of using them, they've become embedded in my mind. On the Pre, no matter what app you're in, you can simply put your thumb below the screen and slide it to the left to go back one screen. No searching for a back button. This is made possible by the fact that the Pre has a dedicated area below the screen that is touch-sensitive. Nothing is displayed down there, so I'm not confused about whether I'm scrolling within an app or issuing a general gesture. I guess it builds off the idea of the Graffiti area from the original Palm, except you don't write letters there (which I always found very difficult to learn), you just gesture.

I find buttons more difficult to learn than gestures, I think, because it's a more abstract concept. I have to learn that pushing a button has a certain effect, and then remember that. That effect is rarely related to anything about how the button looks, also known as the button's "affordances" in UI-speak. Icons can help tremendously, but can also confuse. Gestures can be different, because they can build on our already-learned, often-practiced, naturalistic interactions with space. They can still be a bit abstract or perhaps more difficult to use than buttons (e.g. Graffiti), but a good gesture, once learned, beats a button press any day for me. On the Pre, to go back a screen, I don't push a special button arbitrarily assigned that function - I just motion backwards with my thumb. Of course, the concept of "back" is somewhat dependent on language (maybe Hebrew writers would prefer to go back by sliding from left to right), but it can always be culture-customized.

Another example of a Pre gesture that I find intuitive is how you can view all the apps that are running. Just slide your thumb up from the gesture area to the top of the screen, and you can see which apps are running. If I move my thumb again from an app to the top of the screen it flicks it off the screen, quitting it. This is built on the metaphor of "cards", where each app is it's own card. I like the metaphor of tossing the card off-screen to quit an app. It's somehow oddly satisfying, perhaps because I like playing card games and it feels a bit like dealing someone a card. If you get a chance, you should try it and let me know if you feel the same thing. The nice "swish" sound also helps.

The last gesture that is unique to the Pre is the way to get to the list of apps from any screen (iPhone users might call this the "Home" screen which comes up when you push the button). The gesture for pulling up the app list is identical to the one for seeing all the running apps, you just do it a second time in a row. It feels like I'm opening a hidden drawer. It feels right, so I like it.

Of course, I'm sure many of the Pre gestures can be improved, and some of them are inconsistent (as Sachin pointed out to me), and some described above may not seem intuitive to you, even after you've had a chance to use it hands-on. But so far, I'm impressed with the Pre's gestures and I think it's the right kind of interface for handheld devices.

The Palm Pre also has a physical keyboard, so call me old-fashioned, but I still prefer it to the virtual keyboard. If I have to enter text, and I have to use a keyboard, give me a physical keyboard any day. There's something about the space-specific tactile feedback, the feeling of not only pushing down on the button but feeling and hearing it spring back. I'm a huge fan of the keyboard on my Thinkpad for this very reason; Apple's MacBook keyboard is pretty good too. Unfortunately, though, the keyboard on the Palm is hidden behind the screen, and to get it out you need to slide the screen away. The physical mechanism isn't very stable, solid, or smooth, and it takes more time to start typing than if the keyboard was just right there in front of me. Also, the keyboard's buttons are smooth, concave, and don't press down very far (little displacement), so it's not actually that much easier to use than a virtual keyboard, because my fingers slide around and I make more mistakes.

The Pixi, Palm's second of two webOS phones, overcomes all of these shortcomings. In fact, I liken it to a streamlined version of my venerable Treo 650. The keyboard has good displacement and may even be better than the keyboard on my Treo. Also, unlike the Pre, the keyboard is always there, though you sacrifice 80 pixels (17% of your screen height) to make room for it (which I think might be worth it). The back is nice and rubberized, so easier to grip. I never understood the appeal of the smooth back of the iPhone - seems like it would slide all over the place and be slightly slippery to hold, though I guess Apple decided that people will buy shinier objects over less shiny ones (which may be true). The Pixi is also really light and thin, and I think it's the best choice out there for people who want a physical keyboard that's always in front of you (though I haven't used it extensively, just in the stores). I've always found the Blackberry hard to use - it seems to have a very specialized interface with a steep learning curve, one that seemed too much work to be worth it to me.

So why don't I just get a Pixi? For one thing, it doesn't have Wifi (which, like the lack of pinch-zoom on Android, boggles my mind). And for another, I can't seem to get Sprint to sell it to me without forcing me to change my phone plan to one with unlimited data that costs $70/month. I'm on a $40/month plan right now, and used to pay $55/month to also get data. I am concerned about the higher cost of data ($360 more per year), and I also don't really want 3G data service. Let me explain.

I had data access on my Treo for several years before the iPhone even came out. At first, I enjoyed reading my email, IMing, listening to streaming radio, and using Google Maps wherever I happened to be. Sure, the network was slower back then, but it was sufficient. After doing this for a couple of years, I realized that I was spending all of this time staring at my phone, when I could have been out in the world, talking to people, looking around, experiencing life. It is definitely super-convenient to have maps and driving directions, and that's the one app that I regretted having to leave behind when I cancelled my data plan. But I found that I was less phone-fixated and more conscious of my world, more aware of what's happening around me and less interested in Googling some random fact. It bugs me when I go out with friends and they spend a lot of time staring at their phones - I guess I must not be very interesting company. Maybe I'm just behind the times, but it's how I feel. I guess in the end, I didn't get enough out of data - it was a fun experiment while it lasted, but ultimately too much of a distraction to be worth it.

So is the desire to have Wifi hypocritical? I don't think so, since I certainly want to be able to install new apps (which have the potential to be useful), and without Wifi or 3G that would be more difficult.

In not wanting 3G data I'm probably in the minority, but what else is new? I can of course get the plan and just not use the data, but though it seems that I have above-average self-restraint, let's be realistic. More importantly, I don't enjoy paying extra for something that I don't use, especially when it's a life-long commitment (since I probably wouldn't be able to switch to a non-data plan later).

My ideal phone would actually be pretty simple - it would be a great device for making phone calls, with a keyboard and a few really useful apps. It wouldn't drop calls, it would have good coverage (Sprint and Verizon-level, if not better), and it would have crystal-clear voice quality, so good that you would be able to tell the difference between someone saying "B" and someone saying "D". Becca's Pre has a pretty terrible microphone, so poor that often it just sounds like she's mumbling if her mouth isn't positioned just right. I think I would pay more than $40/month for a simple cell phone with fast dialing, excellent coverage and superior voice quality, without any data access or even a built-in video camera.

Somehow, in the hyper-competitive arena of cell phones and service, features have won over product quality (and by that I mean *phone* quality). This is especially apparent as the *total* number of apps has become a major marketing factor, regardless of how many of them are actually useful (very few in my opinion, Google Maps being an exception). Maybe one day the momentum will swing back in the opposite direction, and we'll actually have cell phones that dial quickly, don't drop calls and sound as clear as someone sitting right in front of you. Such high quality of service is so foreign to me that it's a bit hard to imagine. But for the moment, let's just try to imagine that.
]]>Comments:
]]>omar the french duke: I type this now on the google phone virtual keyboard. I was very skeptical of virtual keyboards but I must say this one is growing on me. I too am baffled by the lack of pinch. You should fiddle with the new phone when you get a chance. It is vastly improved.

Sachin: Obviously I'm highly biased towards the iPhone. But I will start by saying I too am very impressed by the new Android phones. I think Android will give Apple a run in the coming years and ultimately might win because it's open. (although rumors claim the new iphone in 2010 is the first true redesign, 3G and 3GS were just refreshes. so Android is sorta competing with 2.5 year old tech in terms of software and gestures) Ok, i'm going to battle 2 points to this article (i could argue more) 1. I find the Palm Pre gestures 100% UNNATURAL. I remember saying that the first time i every used one. The various swipes, card metaphor, etc are 100% learned. NO ONE can pick up this phone and just use it. When both my mom and my dad got their iphones, i remember not getting a chance to show them anything. (I think both times I was leaving down right after we went to the apple store). I never had to teach them anything, they got it naturally. And they are old! I think your bias towards the palm gestures is largely because it's the first touch device you've used. That's always going to feel more natural than something you see later. But all the palm gestures are modal, which i find to be terrible. you have to swipe to enter this "card view" before you can swipe up cards to quit. Having these modes will confuse normal people. What mode am I in? Why does this gesture work in mode X but not mode Y? 2. The keyboard. You really can't diss a virtual keyboard unless you've used it for like a week and gotten used to it. I can type so freakin fast on my iPhone, much faster than I've seen people type on their bb. now when I do use a BB or other physical keyboard phone, it's seems too laborious. Having to push down hard enough for the physical click slows me down. If i'm already touching the key, register it and let me move on. And then correct for my mistakes. It's such a better experience. And of course the touch screen gives you more free screen space, but that's a separate issue. It is nice that the keyboard itself can change based on what you are typing. web address, email address, phone number, they each have their own layout. I think optimization of keyboard layout and correction on touch screen phones is going to be huge in the coming years. and on a random aside, switch to posterous, dammit. You'll get way more page views, and i'll be notified of new comments via email. and all the other random stuff...

Sachin: You need a lot of faith to type a long comment here. I kept mine on my clipboard just in case it didn't appear in a few minutes :)

omar: nikhil's blogging technology is so last decade

nikhil: actually, my blog uses technology circa 1998. so it's actually from 2 decades ago!
]]> Write Comment

Tugging Bubbles in a Box

nikhil — 2009-12-15T10:15:18-08:00

Last weekend, I went to San Francisco for a friend's wedding. While there, I was thrust back into a world I left 3 years ago, the world that is my California. It is a world of engineering and of business, the world of working and thinking in Silicon Valley. It is a world raft with rapid, superficial, constructed, and possibly irrelevant, change. And it was one I chose to leave, for many reasons. But one reason I think about now has to do with tugging.

During survival school, I was apart from society for 28 days. I wasn't completely alone, but with a handful of students and a dribbling of guides. I've written about my experiences before, so I don't want to rehash that. What I realize now, and I did a bit at the time, was what impact that loneliness had on how I think about my mind.

While out there, the perturbations to my mental trajectory were slight. Sure, some of the time I talked with the other students and guides, but most of the time it was quiet. There was nothing there but desert and mountain, and my mind was on its own. No distractions, no ads, no one inserting their own thoughts into my head under the auspices of self-realization. Thought interference was at an all-time low.

Now, I'm not reminiscing because I want to go back - God knows that the deprivation of it all was rough, and its primary benefit to me was the return of contrast to a life that I had habituated to. What I am reminiscing about, or maybe what I'm beginning to realize, is that I miss *that* part of it. I miss the lack of distraction. I miss the primary independence of my thought trajectory.

The thoughts I thought while I was on survival school were the culmination of days of independent mental and physical momentum. Sure, the few other people and the harsh environment played some role, but the bulk of thrust was my own.

I'm beginning to think that my mind is like a bubble in a box. The universe of mental states is the box. My mind is but a lowly bubble. And every influencer, that is every other person, every book, every communication, every advertisement, every dizzying distraction, is also a bubble. They come in all sorts of shapes, colors and sizes, all with varying degrees of bubble-tugging power. We're all actually magnetic bubbles, our very presence alone pushing and pulling all those other bubbles around us. But pushing and pulling aren't the only forces in this world - each mind's bubble also has its own momentum and direction, equipped with a veritable engine and rudder. It is this pair that gets us to the places we're supposed to be, this pair that actually defines who we are as independent beings.

My problem isn't that all these bubbles crowd the space, because the space is ever-expanding in both breadth and dimension, far faster than the bubbles actually have the momentum to explore. My problem is that these other bubbles impinge on my own bubble, my own mind, yanking and tugging like magnets on metal, hemming and hawing like seamstresses on upholstery. Often, it seems that these other bubbles just disturb my trajectory or delay my arrival or, even worse, spin me into a new path like a planet would an asteroid. Now don't get me wrong, these new paths can be into parts of the box that I'd love to be in, that I may not be able to enter on my own. But these new paths can also close off whole portions of mind-space that I now will never be able to enter, perhaps portions that I *should* be in. It's a risky place, this bubble-bouncing paradise.

I'm a bit sick of a lot of the other bubbles. I feel like they tug me to places I don't want to be. Not having a TV helps keep a whole herd of bubbles at bay. I want to pull what I want closer, push what I don't want farther, and get close enough to opinionize all that lies in between.

Maybe this is one reason why I'm enjoying grad school so much. Grad school has pulled me into a whole new dimension of the box, the dimension of science (which is unexpectedly very different from the dimensions of engineering and business). It is in this new dimension that there are so many unexamined bubbles, and my intrinsic momentum retains its strength. In this new dimension, unlike in business, what matters is not what the other bubbles think or do. What matters is that the truth lies waiting, rigid, unperturbed by all the bubbles. Somehow I've been able to lock on to this stability, yoking myself to it as a shield to a storm. Things feel pure, feel real, and feel true. The point of my work is not to sell something, or to compete in a public Lego-building contest. The point is to understand that which is true, that which is observer-independent, that which forms the texture of a bubble-empty space.

I oscillate between regretting my psychological isolation and extolling it. You can guess which phase I'm in now.
]]> Write 1st Comment


Left/right	Circle	Circle gradient (default)

Diamond gradient	Corner gradient