Archive for the ‘chumby’ Category

City Pictures from China

Thursday, September 6th, 2007

After going through the many photos I took in my travels over the past several months, I found two that I thought were interesting and shareable. Both images are clickable links to a higher-resolution version; and remember, all content on this website is Creative Commons by-nc-sa.

This photo is a nice juxtaposition of an artery of speeding humanity against the steel-and-glass body of Hong Kong that it nourishes. This was taken from the balcony of the China Club in Hong Kong. The China Club is a members-only affair, and we were fortunate to be there as guests of Liam Casey, aka “Mr. China”.

This was an unusually sunny day for Shenzhen; in honesty, it’s rare to see blue sky due to the pollution in the area, but for several days in a row during that trip there was beautiful weather. Shenzhen’s skyline is growing at a frenetic pace, and it’s a very planned city, as evidenced by the regularly arrayed skyscrapers–it’s almost as if the city planners played way too much Sim City in a previous life. Shenzhen city is slated to have a minimum of 40% “green areas” and so far they are doing a good job of keeping to that. The ever-present construction cranes in the foreground are an indication of the eternally under-construction look of the area. This picture was taken from the front of the new Sheraton Futian Sheraton, a hulking, obsidian black palace of a hotel, a gaudy icon of parvenu Shenzhen. Comfortable rooms, so-so food (but good food can be found in walking distance) and as always, free bottled water and free internet (a custom I wish US hotels would adopt!).

More Scope Pr0n

Thursday, September 6th, 2007

The Youscope demo is a hard act to follow up on, but I’ve had this scope screenshot for a while now and I thought it was so neat that I wanted to talk about it a bit on the blog.

Before I dive into the post, let me say that the best accessory I ever bought for my scope (a Tek TDS5104B–don’t let anyone ever tell you an Agilent is better than a Tek!) is the P6245 active probe. After suffering for many years with a Kikusui 100 MHz analog scope with second-hand passive probes, the quality of measurements I get with the active probe and the TDS5104B brings a tear to my eye. For the first time ever I can see traces that actually look like the darn SPICE simulations.

Here’s the background behind what we’re looking at in the screenshot. The top trace is the data line of a memory bus; the bottom line is the clock. The trigger is set on the data line–not the clock line as you’d expect. The test pattern being run is a repeated 0xFFFFFFFF-0x00000000 transition using four different bus drive strengths in sequence on the microprocessor. You can see each of the four drive strengths on the bottom clock trace, for example, at point “C”.

The particular bug this trace is supposed to capture is a ground bounce problem, as shown at point “B”. The rising transition “F” is what causes the ground bounce at “B” (note that the dip at “B” does not happen on anywhere else). The reason “F” trails the ground bounce event is because the cable of the passive probe used for the Data line has a fairly long delay–another measurement was used to calibrate out this fixed delay. The time difference of F – A approximates the delay differential between the active and the passive probes. You can also clearly see the rise time contrast between the two probes in this screen shot.

I think it is so cool that one can so clearly capture a problem as difficult to characterize as on-chip ground bounce using only conventional probes on external signals.

The other really neat thing about this set of scope traces is that it shows the timing of both the microprocessor->memory and the memory->microprocessor in a single shot. This is because I’m triggering off of the data line, not the clock line. On the data trace, you can clearly see the Processor-driven data trajectory (“D”) and the Memory-driven data trajectory (“E”). On the clock trace, you can see two phase-shifted versions of the clock. One of them is the clock timing relative to when the CPU drives the bus–this is G-F. The other one is the clock timing relative to when the Memory drives the bus–this is C-F (minus a clock period). So in a single picture, you can divine the available setup/hold margin in both directions of the bus!

You can also see other “good things” in the scope trace, such as the auxiliary measurements for frequency, amplitude, duty cycle, and rise time. You can also see how the clock trace is pretty well formed overall, with minimal over/undershoot, and you can get an idea of how much a passive scope probe introduces overshoot artifacts by contrasting it against the top trace. This picture is truly worthy of a thousand words!

OK, so I’m a real geek for getting so excited about a screen shot like this, but really, after teaching this sort of stuff for several years at MIT and then running countless simulations of chips to validate scenarios like this, it’s somehow very satisfying to be able to go into lab and actually see that the real world does match up with theory so nicely (even if it is a bug). If I ever teach digital design again, this shot is going into my slides and my problem sets.

And, yes, this was a problem found on the chumby DVT prototypes some months ago and it has since been resolved. This ground bounce, under certain conditions, would upset the internal clock multiplier of the CPU. The fix involved multiple improvements to the board layout, but in the end nothing can compensate for the relatively high inductance designed into the chip package. Therefore, the most important fix was to use a much higher clock frequency reference so the multiplication factor was only on the order of 20x instead of 10,000x (a 16 MHz reference instead of a 32.768 kHz reference). Reducing the period of ground-bounce noise integration by a factor of 500 resolved the stability problems of the internal VCO of the CPU’s clock multiplier PLL.

Name that Ware July 2007

Thursday, August 2nd, 2007

The wares for July 2007 are shown below.

Again, I’m having fun with the X-ray machine. I like looking inside parts that you normally can’t look inside of. The prize this month goes to the first person who can correctly guess the Ware in either image!

I’ve also got some more posts from the Made in China series forthcoming, but it takes a long time to assemble the movies and write the text and I’ve had a busy schedule (since I last blogged I’ve been to China and Portland). Chumby just shipped its first 50 units and we’ve got the full production ramp right on the heels of these units–lots to do! Based on the enormous response Chumby got on the first 50 offer, demand is going to be quite high for chumbys so be sure to sign up on the website to be notified when broad distribution starts, or else you may not be able to get one for a while!

Made in China: Skill

Friday, July 13th, 2007

One of the most remarkable things about working in China is how much skill the workers have out there. I think the video below speaks for itself.

[Youtube link for those who cannot view embedded SWF.]

This guy works at the factories that sew the chumby bags. Apparently, he’s not their fastest employee. They have one who is about twice as fast, and he has been with the company for about seven years. I went to his workstation, but when I got there he was already gone to lunch because he had finished everything. And I mean, there were two enormous bins of finished cosmetics cases next to his workstation.

I think it’s also interesting to notice that the guy in the video above is listening to his iPod while he sews.

Another thing that’s pretty amazing is how rubberized tags are made in China. These are the tags you see all over clothes–chances are you are wearing a piece of clothing or you carry around a bag with a tag like this. I always thought that the tags were pressed by a machine.

I was wrong. All those words, colors, and letters–they are drawn by hand.

Amazing.

[Youtube link for those who cannot view embedded SWF.]

I asked PCH if they had any mechanized factories for this kind of stuff. They told me that they exist, but the minimum order quantity is enormous–hundreds of thousands, sometimes millions–because of the extraordinarily low cost of the product and the relatively high cost of the tooling for the automated process.

This is consistent with a comment someone made to me once about the McDonald’s Happy Meal toys. If you look at the bottom of one, it’s held together with screws. That’s because it’s cheaper pay someone to screw together that toy over the whole production run for it than it is to make a steel tool with the necessary precision so that it just snaps together.

There is a similar trade-off inside the Chumby hardware. There are four connectors on the internal chumby electronics. One had a best price of about $1 US, and the other three had a best price of about $0.40 US each–using the US-based vendors that I could source. PCH’s very talented sourcing expert (she has a reputation that is feared and respected by every vendor) managed to find me connectors that cost $0.10US and $0.06US respectively–saving almost a full $2 in cost. There’s one catch: these connectors don’t come with the sacrificial plastic pick and place pad that enables them to be machine-assembled.

The solution? Witness the man below.

On every chumby, he hand-places these connectors, for about a nickel per chumby.

Thanks to him, chumbys are $2 cheaper–which frees up more money for us consumers to spend on $2 coffees at Starbucks.

Made in China: Dedication

Friday, July 13th, 2007

This story needs a little background to fully appreciate.

There is a microphone in the new chumby. The particular microphone I decided to use has an integral pre-amp FET (an “electret” type). As such, it must be inserted in the correct orientation with respect to the circuit so the FET receives a proper bias current.

The first samples I got back from the factory had the microphone in backwards. So I called the factory and told them that they need to reverse the polarity of the microphone. I was going to come in for a visit the next week, and I wanted to see corrected samples. When I got to the factory and tested the microphone, I found out to my dismay that the microphones were still not working.

How could this be? There are only two ways to put a microphone in.

It turns out that they had two operators on the line assembling the microphone. One solders the red and black wires to the microphone. The next solders these red and black wires onto the circuit board. The operators were told to reverse the order, and both of them dutifully complied…giving me a microphone that was still soldered in backwards, but with the color of the wires swapped. This is actually a pretty typical story for problems in China…

At any rate, this leads up to the real point of this post. The next day, we had a first pilot run of 450 circuit boards scheduled up. Everything had to go perfectly for us to be on schedule. We had stencils rebuilt (we were debugging a yield issue with the QFN packaged audio CODEC as well) and ready by around noon, and around 6PM I had the first boards in my hands to test. As I was running the final factory test, the device failed again–at the microphone.

Needless to say, this was not a happy moment for anybody in the factory, as the factory is liable for any manufacturing defects. I donned my smocks and marched onto the line to start debugging the problem.

That’s me at 3 AM later that day. I’m still in the factory, and so is every manager and tech involved in the project. The pressure was fairly enormous–right next to us was a line churning out 450 potentially defective circuit boards, and I was unwilling to pull the plug on it because I didn’t know what the root cause was yet, and we had to stay on schedule.

I literally had a panel of factory workers standing by me the entire night to help me with anything I needed–soldering irons, test equipment, more boards, X-ray machines, microscopes. The remarkable thing is that not a single one hesitated for a moment, not a single one complained, not a single one lost focus on the problem, people cancelled dinner plans with friends without even batting an eyelash. If they weren’t needed that moment they were busy overseeing other aspects of the project. And this went on until 3 AM. With one exception, I hadn’t seen blind dedication like this since I worked with the autonomous underwater robotics team at MIT.

Embarassingly, the problem wasn’t their fault in the end. It was the new firmware release that was given to me earlier that day by the team in the US–it had a bug that disabled the microphone due to a hack that was accidentally checked into the build tree.

I think even more impressive is that when they found this out, nobody was angry, nobody complained (well, the sales lady gave me a hard time but I felt I deserved it; nevertheless, she was kind enough to accompany me on the production line all night long and be my translator, since my Mandarin isn’t up to snuff). They were simply relieved that it was not their fault. We all parted ways and I came back into the factory the next day at 11 AM after a good nights sleep. I saw Christy and I asked her when she came in. She told me she always has to report in by 8 AM. Now, I was starting to feel really bad–she stayed up late because of our bug and she came in early while I slept in. I asked her why she stayed up so late even though she knew she had to report to work at 8 AM–she could have gone home and we could have continued the next day. She just smiled and said “it’s my job to make sure this gets done, and I want to do a good job.”

On the right is Christy (PM), and to her left is Xiao Li (QA manager). The equipment they are observing is the chumby production tester, a bed-of-nails device developed in the US by me that facilitates the automatic firmware programming, unique keying, and testing of every circuit board.

Here’s another interesting story. On our way out of the factory floor one day, Xiao Li asked me what does a chumby do? Well, I don’t speak chinese very well, and she doesn’t speak english very well either, so I decided to start with a few basic questions.

I asked her if she knew what the world wide web was. She said no.

I asked her if she knew what the internet was. She said no.

I was stunned. Here is a girl who is an expert in building and testing computers–I mean, on some projects she has probably built PCs and booted Windows XP a hundred thousand times over and over again (god knows I heard that darn startup sound a zillion times that night on the factory floor, as right next to me was a bank of final test stations for ASUS motherboards)–yet she didn’t know what the internet was. I had taken it for granted that if you touched a computer today, you were also blessed by the bounties of the internet. I felt like a bit of a spoiled snob and a pig all at once for forgetting that she probably couldn’t afford a computer, much less broadband internet access. If she were given the opportunity, she was certainly smart enough to learn it all, but she’s busy making money that she’s probably sending back to her family at home.

How do you describe the color blue to the blind? In the end, the best I could do was to tell her it was a device for playing games.