Archive for the ‘Made in China’ Category

Made in China: Getting Started

Sunday, July 8th, 2007

I found setting up a supply chain in China to be enlightening, and I’d like to share more of my experiences with the readers of this blog.

Presenting this material is challenging; there is a lot of detail and its easy to get overwhelmed and lost in the vagaries of electronics manufacturing. In order to focus my writing and create a more succinct body of works, this series of posts will be thematic. In other words, instead of talking about the details of how pick and place machines work in China, I’ll post about a more abstract theme such as “automation” and just show sound bites to give the reader a flavor of what’s going on.

The theme of this first post is “Getting Started”.

In September of 2006, Chumby was just a team of about a half dozen people, and we had given away about 200 early prototype chumby devices at Tim O’Reilly’s FOO camp. The devices were well received by the attendees of the conference, so I got the go-ahead to build the Asian supply chain.

Before we went to China, we had a trusted vendor in the US give us their best quotation for the job, so we had a reference point to work downwards from.

In a nutshell, we called up a lot of our friends who had some experience in China, and we lined up a series of about a half dozen factory tours in China. We hit quite a variety of places on our tour, from specialty factories as small as 500 people to mega-factories with over 40,000 people.

Being open source helped a lot in terms of kick-starting the process, because first we had no fears about people stealing our design–we’re giving the design away–and there were no NDAs to sign when sharing critical information, like the bill of materials. I think this kind of openness actually gave us a better reception with the factories in China; they seemed more willing to open up to us because we were willing to open up to them. Also, there was no question in the factory’s mind that they were in a competitive situation. It was plain that anybody could and would quote and bid on the job (in fact, we received a few unsolicited quotations that were quite competitive), so it saved a round of huffing and puffing. And finally, it seems that despite handing out our BoM without an NDA, nobody in the general public is aware of the strategic improvements we rolled into the chumby hardware since the FOO prototype (then again, chumby has no serious competition at this point in time).

In November, Steve Tomlin and I went to China to do the factory visits, and some of my first impressions were documented in my post Adventures with the Venture Communist.

There is no substitute for going to China to tour the factory. Pictures are always deceiving, and you just can’t get a sense for the scale and quality of the factory without going to see the actual facility. In general, factories welcome you to take a tour–I wouldn’t work with one that didn’t allow me to come visit. However, most factories do appreciate a week prior notice before you come and visit, although as your relationship with the factory progresses things should become more open and transparent.

While almost every factory will “clean up” the day you come to visit, a sharp eye and the right questions can see through any quick veneers put in place. One place I always liked to visit was the QC room. I expect to see rows of well-maintained and well-worn binders with design documentation and QC standards, as well as “golden samples”. I would point at a couple of random binders and demand to view their contents, see the product’s golden sample, and verify that they knew what was going on in the binder–that it wasn’t just some random data they threw in there. Also, hard investments in equipment is a good sign: the best manufacturers I visited all had a couple rooms with sophisticated equipment for thermal, mechanical, and electrical limit testing, and of course operators were in the room actually using the equipment (I could definitely believe a Chinese manufacturer would buy a room of equipment just for show and not actually use it).

After reviewing several manufacturing options, we decided on one, which was PCH China Solutions. PCH itself owns only a few facilities, but it has a comprehensive network of trusted and validated vendors–primarily in China but also Europe and the US as well. Not surprisingly, the factories that PCH sub-contracts to were some of the best facilities we visited while we were in China.

PCH is actually headquartered out of Ireland, and as such, most of their staff engineers are Irish, so there was no langauge barrier. They are hard working, resourceful, well-trained, and as a bonus they always seemed to know the best places to find a pint no matter where we are. I had no idea China had so many Guiness taps.

In retrospect, this is definitely the way to go–I learned a lot about what it takes to do business in China by shadowing their engineers while I was out there. From 10,000 miles away it looks easy but it’s anything but easy working with even the best Chinese contract manufacturers. I’ll cover some of the “frustration stories” in another themed post later on.

Shenzhen Diary, April 2007

Saturday, May 5th, 2007

I’d thought I’d write a brief (okay, it ended up much longer than I thought it would…) blog entry about some of my most recent experiences in China.

I’m currently out in Shenzhen working on bringing up the Chumby production line. This means everything from raw material approvals to programming, testing, QA, and certification for not only the Chumby hardware itself but also its accessory line. It’s a lot of work.

I’m very lucky to be working with a fabulous company out here which serves as my liason between vendors and contractors. I’ve been told explicitly by Chumby that I can’t mention the name of this company because it is a “strategic” advantage for Chumby at this point, but I also made it clear that eventually I will talk about this company publicly, after the Chumby hardware is launched and the magnitude of the strategic gain from radio silence is reduced. I’ve got some nice photos of Chumbys in various states on several production lines that I’d also like to share as well someday.

Shenzhen is quite an up and coming city. I’m staying in a serviced apartment on Xinwen road called the Frasier Futian. It’s cheaper than a hotel yet it has similar amenities (such as maid and laundry service and complementary breakfast), although the quality control of the details of the facilities isn’t quite the same as at a name-brand hotel. I’d recommend this as a place for anyone who needs an extended stay in Shenzhen. The location is good–several very nice restaurants in walking distance, a Starbucks and some convenience shops–and it’s about a 5-10 minute walk to the MTR (the subway system in Shenzhen–you can go all the way to Hong Kong on the subway from here). Most importantly, the internet works great here; I have no trouble using Skype and even video conferencing works well.

The most miserable two days of my life were also spent around here. I got food poisoning–I think it was from handling money (which is filthy dirty out here, it literally smells of human waste) and then eating my food, because I’ve been reasonably careful to avoid anything that’s uncooked here in Shenzhen. It was the worst food poisoning I’ve had in my life, it lasted five days until I finally broke down and went to a doctor in Hong Kong and got antibiotics for it (I’m told you never want to go to a doctor in Shenzhen if you can help it). The doctor gave me Ciproflaxin–which I have an allergy for–but the doctor was like, “do you get a rash around your mouth?” and I was like “no, only on my hands and feet”, and she was like, “well, take the Ciproflaxin then. It won’t kill you and you’ll need the strength of Cirpo to get rid of those bugs in your gut.” It worked; I got better. I did get that rash, but I also didn’t die. One really nice thing about the doctors offices in Hong Kong is that they also fill your prescription at the office, so I didn’t have to run around Hong Kong searching for a pharmacy. I was diagnosed and taking my first dose of antibiotics all within 15 minutes (and at a fairly reasonable price–about $125US for a weekend visitation, including the medications).

Anyways, while I had the food poisoning and before I could see the doctor, I had to go to a factory in Donguang at 8AM and I was up until the next morning at 3 AM debugging the manufacturing problems in the very first run of the Chumby circuit boards coming off the line. The fever, cramps, dehydration and constant trips to the toilet didn’t help progress. I spent the night in the factory dorms (where the workers also stay), which isn’t too bad of a place (actually a bit nicer than many of the dorms at MIT) but I ran out of toilet paper around 5 AM (I was making trips to the bathroom every hour) and I had to ehm–improvise. There is no concierge service in a factory dorm! The next morning I had to get up at 8 AM again and continue debugging. Fortunately, most of the significant problems were resolved and the root cause of all the remaining issues are at least understood and solvable in time for production (or so I hope!). There are more interesting stories around the root cause of some of these issues, but let’s save them for another day.

In the end, all I can say is what doesn’t kill you makes you stronger…and I also carry around a bottle of hand disinfectant now for after when I handle money. Surprisingly, it was about the hardest thing to find in the drugstores out here. It was buried in with the urgent care first aid area, oddly enough, next to the alcohol used for disinfecting wounds. I think the store clerks didn’t understand that this product is for preventative use, and not for urgent care use.

I also had a little adventure finding a fedex out here. Normally, fedexes aren’t hard to find, but there was a three-day vacation called the “5-1” (for May First). It’s a holidy declared by the communist party to celebrate the efforts of the laborers. Fedex was closed for three days and I urgently had to send a package. So, I hopped on a boat to Macau because I knew the casinos would be open on May 2nd and I couldn’t imagine a high-end casino not having a business center with a Fedex. And indeed, the Wynn Macau had a fedex with english speaking staff that happily processed my package on time!

Macau was also a very interesting place. There is a lot of hype about Macau being the next Las Vegas, with casinos bigger and more fantastic than Vegas. Macau casinos were certainly opulent and impressive, with a great many gimmicks and displays, but I think the hype is a little bit too much. I’ve been to Vegas many times, and currently, I think Vegas still trumps Macau. At least, the food in the Vegas casinos is better, and the clientele is more fun. Let me tell you, gamblers in Macau are hard core. In Vegas, you see people at least smiling and having fun at most tables. In Macau, even the simplest of dice games had chain-smoking men staring intently at the table. Nobody drinks alcohol–they all have coffee or tea, to keep their minds sharp. You can feel the intensity–the casinos are eeriely quiet compared to the din of the Vegas casino. I’m not that into gambling, so I guess it wasn’t for me.

I thought the local food in Macau was quite good. The Portuguese egg tart is tasty–it’s a variant of the Chinese dim sum called “dan gau” (simply put, an egg tart). While the Chinese version tastes quite eggy, the Portuguese version uses a type of flan for the interior, so it’s mildly sweet and has a nice texture to it. I also ate some local Portguese food, which was very nice. It’s very similar to a Japanese curry, I thought, although with less curry and a slightly more watery sauce. You can really taste the flavor of the meat through the sauce–I had the oxtail and chicken. I have no idea what it’s called, I just asked the waiter what was popular. Incidentally, I wasn’t as impressed with the high-end casino food.

Speaking of food, I also went to the most fantastic noodle shop in Hong Kong today (I had to run down there to shop for fabric samples for the Chumby–they have a great textiles market in Sham Sui Po). It’s the Crystal Jade–apparently a chain restaurant–I went to the one in the Harbour City Ocean Center. They have hand-pulled noodles there and oh man they were so good. The Harbor City Ocean Center is near the Tim Sha Tsui MTR stop, just stop into any hotel near the MTR station and ask the concierge for directions.

This post has grown much longer than I thought it would be, so I’ll stop the story telling now and leave you with these two pictures of street signs that I thought were just…interesting.

Obviously, the no bikes sign isn’t obeyed. Makes you wonder about the other ones.

Where Have All the Innovators Gone?

Tuesday, February 13th, 2007

People have often asked me, now that I have some perspective on China, what I think will happen to the US. Can we compete? Will we continue to lead? I’m quite bullish about the US in general, but I had an interesting reality check tonight. I’m at ISSCC 2007 right now (where I and my former colleagues at Luxtera had the honor of receiving an “outstanding paper” award for work presented at last years’ conference), and I was chatting with UCSD high speed integrated circuits professor Jim Buckwalter about the nature of the graduate student applications he has received.

The statistics were astonishing. Of the thousands of applicants, only 80 were from the US. To put this in perspective, he had more applicants with the surname “Lee” alone than he had domestic applicants. And UCSD engineering is no slouch; according to the rankings they are #11 in engineering overall. Even more interesting is that apparently Korean students studying in the US get Korean-government sponsored fellowships–clearly that gives them an edge when considering who to take into your graduate program.

The enormous disparity in domestic applicants to higher education in crucial fields such as high speed circuit design is a bit disturbing. With numbers like these, it is inevitable that the US will lose its edge in technology. I guess it wouldn’t be as bad if these foreign students actually stayed in the US and started companies, but my experience in China has shown that just about every company I talked to had US-educated management from schools like Berkeley and Stanford.

Now, a protectionist mode of thought would suggest that we should put quotas on the number of foreign people we admit to our universities. That doesn’t work because US citizens don’t want to go to graduate school in electrical engineering, as evidenced by the paltry showing of domestic applicants, and forcing them in doesn’t make us more competitive in the global sense.

As the son of Chinese immigrants born and raised in the cornfields of Michigan, clearly I’m disposed to argue that we should try harder to woo these brilliant foreign minds to graduate and set up shop here in the US. Back when my parents came, staying in the US was an easy decision, because China was not a land of opportunity. But in this new global economy, the US no longer has the monopoly on opportunity. That’s the big paradigm shift here that I think we aren’t internalizing. We are no longer “the land of opportunity”– we’re now just one of the better places to find an opportunity.

When you don’t have a monopoly, it means there is competition. We need to compete to retain foreign talent, but instead, we hassle them away. I just wrote a green card recommendation for a brilliant photonic circuit designer. It seems weird that he has received such scrutiny and is going through such detailed background checks when anyone who lives in a border town like San Diego knows there is another easier way for immigrants to sneak into the country and make a living–and I have a feeling the guys sneaking in don’t have PhDs in electrical engineering. And it really bugs me that a brilliant Iranian circuit designer friend of mine just got interrogated by the FBI out of the blue, but presumably motivated because of current events in the world. He’s not a terrorist, and he doesn’t make nukes, despite his Farsi-sounding name. He is a core technical contributor in a US electronics company whose work has been critically peer-recognized as innovative and valuable. We should be rolling out the red carpet for these innovators, and not making them feel like aliens.

While I understand the motivations of many of our immigration policies, it is becoming clear to me that in practice, something is broken here, and the loser will be the US. The beauty of a melting pot is that we have the opportunity to incorporate the best and brightest minds into our culture; instead we skim the cream and throw it away, simply because they are the easiest and most cooperative targets. The system is hassling the people who are educated, and rewards those who are not. This is because the educated ones know the rules and are held to a high standard, and those who don’t know the rules often are not well educated so they have well-meaning public advocates who try to represent and defend their interests.

Of course, these privileged foreigners don’t need public advocates, and they don’t make a fuss, so their problems rarely garner the attention of the public eye. They are resourceful, self-sufficient, and they have other options–if the US gives them the run around, they can always take their good ideas and start a company back at home.

Akihabara, Eat Your Heart Out

Wednesday, January 31st, 2007

Ten years ago, Akihabara was the place to be for the latest electronics and knick knacks and components. I’m convinced the new place to be is the SEG Electronics Market in Shenzhen (although to be fair I heard there is a competing market in Korea that’s supposedly even better–the Japanese test-market their stuff there even before they try it in Akihabara!).

As I first step foot into the building, I am assaulted by a whirlwind of electronic components. Tapes and reels of resistors and capacitors, ICs of every type, inductors, relays, pogo pin test points, voltmeters, trays of memories, all crammed into tiny six-by-three foot booths with a storekeeper poking away at a laptop, sometimes playing Go, sometimes counting parts. Some booths are true mom-and-pop shops, with mothers tending to babies and kids playing in the aisles.

Other booths are professional setups with uniformed staff and work like a bar for electronic components, complete with bar stools.

And it’s not like, oh, you can get ten of these LEDs or a couple of these relays like you do in Akihabara. No, no. These booths specialize and if you see something you like, you can usually buy several tubes, trays or reels of it–you can go into production the next day. Over there, a woman sorting stacks of 1GB mini-SD cards like poker chips; here, a man putting sticks of 1 GB Kingston memory into retail packages, next to him, a girl counting resistors.

Stacks of power supplies, varistors, batteries; ROM programmers. Atmel, Intel, Broadcom, Samsung, Yamaha, Sony, AMD, Fujitsu, every variety of chip. Some of them clearly ripped out of used equipment and remarked, some of them in brand new laser-marked OEM packaging.

Chips that I couldn’t dream of buying in the US, reels of rare ceramic capacitors that I only dream about at night. My senses tingle, my head spins. I can’t supress a smirk of anticipation as I walk around the next corner, to see shops stacked floor to ceiling with probably a hundred million resistors and capacitors.

Oh my god! Sony CCD and CMOS camera elements, I couldn’t buy those in the US if I pulled teeth out of the sales reps–and behind the counter, the guy sometimes has a datasheet–ask for it. A stack of Micrel regulator chips–over there, a Blackfin DSP chip for sale. The smell, the bustle, the hustle. It’s the ultimate electronic component flea market. Over here, a lady counting 256 Mbit DRAM chips…trays of 108 components, stacked twenty high, a row of perhaps 10 of them–she has the equivalent of Digikey’s entire stock of DRAM chips sitting right in front of me.

And across from her is a half dozen more little shops packed with chips just like hers. A man standing proudly over a tray of 4 Gbit NAND FLASH. All of this available for a little haggling, a bit of cash, and a hasty goodbye. This is Digikey gone mad. It’s as if they let the monkeys into the warehouse at Thief River Falls, Minnesota and spilled it into a flea market in China, and then some.

And that’s just the first two floors. Six more floors of computer components, systems, laptops, motherboards, digital cameras, security cameras, thumb drives, mice, video cameras, high end graphics cards, flat panel displays, shredders, lamps, projectors, you name it. On weekends, “booth babes” dressed in outrageous Acer-branded glittery body suits are loitering around trying to pull you in to buy their wares. It’s got all the energy of a year-round CES meets Computex, except the point here is not to show off the latest technology–it’s to get you in to these booths to buy it. Trade shows always feel like a bit of a strip tease, with your breath making ghostly rings on the glass as you hover close over the unobtainable wares underneath. This is no strip tease. This is the orgy of consumer and industrial electronic purchasing, you can get your grubby paws on every piece of equipment for enough quai out of your wallet.

A brisk walk down the street 3 blocks lies the Shenzhen bookstore. The first and most visible rack of books is a foreign book section, packed with classic books like Thomas Lee’s RF design book and several Razavi titles. I pick up Lee’s book…68 quai, or $8.50. Holy cow! Jin Au Kong’s book on Maxwell’s Equations…$5. Jin Au Kong taught me Maxwell’s Equations at MIT. I go on a spree…I pack my bag with six or seven titles, probably around $700 worth of books, and I go to the checkout counter and buy them for less than $35–complete with the supplemental CDs.That’s like an economy class ticket to Hong Kong right there!

Knowledge is cheap. Components are cheap. The knowledege in those books are the Real Deal, and the parts down the street are all there. And within an hours drive north is probably 200 factories that can take any electronics idea and pump them out by the literal boatload…and these are no backward factories. I saw with my own eyes name-brand 1550nm single-mode long-haul fiberoptic transcievers being built and tested out there. Shenzhen is fertile ground. You need to come here to see it to understand it. As a technologist from the US, I tremble in my boots, with terror and excitement–I get to be a part of this! This place has the pregnant feel of the swapfests in Silicon Valley back in the 80’s, when all the big companies were just being founded and starting up…except magnified by 25 years of progress in Moore’s Law and the speed of information flow via the Internet. In this city of 12 million people, most involved in tech or manufacturing, plenty of foreign influence, many learning English, all of them willing to work hard, there has to be a Jobs and Wozniak somewhere, quietly building the next revolution.

Okay, so this wasn’t a name that ware…it’s lame, but my dog ate my homework. I lost my digital camera on the way to Shenzhen in the plane somewhere, so now someone has a camera full of pictures from Christmas, factories, bachelor parties, and idiosyncratic amounts of close-up shots of electronics. The guy who found it has to think I’m a weirdo (I guess I can’t deny that!). I’ll find another ware and put it up soon–give me a couple of days!

Real Sichuan Food

Wednesday, January 31st, 2007

Despite growing up in a Chinese family (granted, a Chinese family in the midwest of the USA), until last week, I was never exposed to real Sichuan food. Sure, sure, I had sampled many of the restaraunts in various cities in the US, but typically the cuisine was Cantonese, Shanghainese, Taiwanese or at best Hunan. Eating real Sichuan food in China (yes, I was back there again!) was a bit of an experience.

Soo…above was a chicken dish I had. Well, chicken is sort of applied in the loosest of ways. It’s more like chicken hidden in a sea of red peppers. You sort of pick your way through it, looking for morsels of chicken, and the bolder ones of course just eat it all. But that wasn’t the most interesting thing.

Above is the real Ma Poa Tofu. It looks innocuous enough, but to my surprise, there is a spice I have never had before. My hosts couldn’t translate the English name to me, but it turns out to be, according to Wikipedia, the Sichuan Pepper.

That’s it up there. In Chinese, the word for “spicy hot” is la. The word to describe the effect of the spice pictured above in Chinese is ma–and that’s the “Ma” in Ma Poa Tofu that I had been missing all my life (nice how ma and la rhyme, huh? makes for some poetic sounding dinner conversation and cheeky sayings about the Sichuanese in Mandarin). Many of us are familiar with the la sensation–that burning, five-alarm sensation in your mouth. Ma is something else. Your lips tingle and numb. Your mouth tingles. As you swallow it, your stomach feels strangely warm and tingly all over. It’s definitely spicey, but it’s not hot…it’s…ma. I think the ma sensation fights the hot-spicey sensation so you can eat more of it. Or something like that. But everything they say about how hot Sichuan food is, is true. And anecdotally, just as addicting. My friend was telling me that his friend’s girlfriend is from Sichuan, and apparently if she doesn’t eat at least one scorchingly hot meal a day, she gets extremely irritable and very aggressive.

I’m wondering how in all my years eating Chinese in the US, I have never come across this spice. The Wikipedia page notes briefly that the FDA banned it from 1968 to 2005, which may have something to do with it.

You learn something new every day. Speaking of learning new things, did I ever mention that the Irish slang for an unstable object is “wonky” (for example, a table with uneven legs is a wonky table)? I learned that too on this trip, from a joke this Irish girl told me at a pub: Q: What do you call a three-legged Donkey? A: A Wonkey. Q: What do you call a three-legged donkey with one eye? A: A winkey-wonky…I tell you, Shenzhen is growing up to be a rather international place!

And another factoid for you: there are seven women for every man in Shenzhen. I have no idea how true this is–my host asserted this to me–but it seems consistent with the ratios I’ve seen in every factory, office, store, and pub in that city.

The Sichuan chef’s arsenal…