TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

Online Education's Dirty Secret - Awful Retention

197 pointsby pkreinabout 12 years ago

61 comments

pamelafoxabout 12 years ago
(Coursera engineer here) It's great to see thoughtful critiques like this.<p>Thankfully, we have a data analytics team now (3 out of our 17 engineers), and they are studying our retention statistics and the factors that affect it. They're also running A/B experiments to see what increases it and getting some interesting results.<p>We do see some big advantages of the timed model for learning, particularly in classes with peer-to-peer grading and evaluations, but there's obviously a big desire for the self-study mode, which is enabled for a few of our classes currently.<p>We're also introducing things like Signature Track, which some students sign up for just to encourage themselves to make it to the end (and it seems to work for many of them).<p>We'll keep experimenting to see what makes students both happy and successful. :-)
评论 #5470234 未加载
评论 #5470388 未加载
评论 #5469966 未加载
henrik_wabout 12 years ago
Taking a Coursera course is <i>hard work</i>. I've taken 3 courses so for (one on databases and two on algorithms) over the past year and a half. This is about as much as I can manage (I work full time and have a family), even though I would like to take more (so many interesting to choose from). But the main reason for this limit is the amount of work it takes, not how the course is presented, or how well written the mails are.<p>For me, the pace of the course is actually a plus. If there were no deadline, I simply would not get around to doing it. I remember when I first found all the MIT courses on-line (several years ago). I really wanted to take some of them, but because there was no schedule and no deadline, I never got around to taking any of them. It's only with Coursera that I have actually taken (and finished) any.<p>I've written about my experience of all three courses on my blog. The latest was Algorithms: Design and Analysis, part 2 <a href="http://henrikwarne.com/2013/02/18/coursera-algorithms-course-part2/" rel="nofollow">http://henrikwarne.com/2013/02/18/coursera-algorithms-course...</a>
nbouscalabout 12 years ago
I don't see how something can be a "dirty secret" when everyone already knows about it. This is not news, and I think the author is well aware of that and just wanted an eye-grabbing headline.<p>As for courses being too fast: they're college courses. That's a core part of their value proposition. If they were slower, or did not follow a rigid schedule at all (like the "better" examples presented in the article), they would be a fundamentally different product. The author simply doesn't understand the concept behind MOOCs, and probably isn't their intended audience. He would be better served watching lectures on Youtube (I don't mean that sarcastically, there are fantastic courses available there).<p>Synchronous learning is not, as another poster claims, an anachronism. It simply isn't necessary or valuable for everyone. For some, myself included, it is a significant benefit. That is the market that Coursera and edX are targeting, and one shouldn't criticize a company simply because one isn't a member of their target market.
评论 #5472112 未加载
manifoldabout 12 years ago
My problem was mainly the availability of the materials. Codecademy and duolingo give you access to as much as you require from the start and you can go through as quickly as you like. The university driven sites limit access to so much per week (though I'm not sure how courses will operate the second time) and demand you stick to their schedule, though granted this may be due to their need to peer-review the more demanding assignments.<p>Unfortunately my free time isn't available in nice predetermined six-week chunks, but even if I am able to catch up three weeks or more in a weekend the courses gave a very negative vibe about continuing to progress as soon as you miss a single one of their deadlines (i.e.- "you missed our deadline for this multiple-choice computer marked test, so your effort no longer counts"). I've 'failed' several coursera sessions in the fourth or fifth week for that reason.<p>Timetabling seems a very traditional educational view, and it contrasted sharply with codecademy and sites like duolingo where I spent Jan and Feb learning the basics of new languages - computer and human. I finished the courses I took because I did them at my own pace.
viveutvivasabout 12 years ago
The way that Udacity does it is perfect for me -- work through at your own pace. Coursera has a couple of self-paced classes, I believe, and I wish that more of the courses had that option.<p>I think the majority of MOOC course developers want to run their online courses as closely to their university versions as possible, since that's what they're used to working with. It also provides a handy way for them to go off-duty, in a sense, if the course has a finite end date. Using their current pacing structure (which is incredibly difficult, as the OP points out, for people that aren't full-time students) allows the teachers to do something other than devote themselves solely to the course, assuming they don't want to just post an archive and leave it alone -- which would meet a lot of people's needs, but misses the whole teacher-student interaction, which is pretty much missing from MOOCs anyway. If they want to provide an environment that's like a classroom, with students interacting with the instructors and with each other, you kind of need everybody at the same pace. It would be nice for us if they could slow that pace down, but that would probably increase the workload for the instructors.<p>We're still early in this game. I'm glad that so many professors have been willing to invest the time into developing the courses, and I understand why they are currently set up to be conveniently structured for <i>them</i>. I think we'll start to see some improvements if/when the money appears in the MOOC game. Once it's no longer basically charity work for the instructors, there will probably be more efforts to work around student schedules.
评论 #5469797 未加载
spikelsabout 12 years ago
Everyone needs to understand that the low completion rates are not as bad as they at first seem. The way to try out these classes for free is to sign up. There is no obligation at all: completely voluntary, no cost, no time commitment and it will not show up on permanent record.<p>These completion rates are actually more like conversion rates for free trials. As many of us know these are almost always quite low. How many people actually ever read books they "Look Inside" on Amazon? How many people finish long articles on the web? Now many people sign up for paying accounts after free trials of your new website? Not very many.<p>They are not the same as dropout rates in high schools or universities despite what some online education haters say[1]. Like any disruptive technology the existing players are threatened so you need to pay attention to the source of the criticism.<p>Instead completion rates are a metric that can be used to improve the class such as the suggestions here. The developers of online classes should and are using A/B testing to improve completion rates. However like almost all conversion rates they will likely remain low.<p>[1] <a href="http://mitsloan.mit.edu/shared/ods/documents/High-Costs-of-Free-Online-Education.pdf&#38;PubID=5082" rel="nofollow">http://mitsloan.mit.edu/shared/ods/documents/High-Costs-of-F...</a>
UnoriginalGuyabout 12 years ago
That's because online education tries to simulate traditional education.<p>Back awhile ago free online lecture videos (e.g. Berkeley) were simply uploaded to a page and anyone could go watch them without signing up for a class or otherwise jumping through hoops.<p>This resulted in me watching a few random lectures about subjects I know little or nothing about. Like econ classes, engineering, science, or similar.<p>This was good because it was short bursts of information without any hassle, commitment, and similar.<p>These days everyone has locked their content behind sign up/registration pages, and you're expected to commit to a tradition semester of a class in order to get a worthless certificate of completion.<p>They also dictate the speed at which you learn. No more learning at your own pace, no, you have to do one a week for as many weeks as it goes on for like it was a traditional class.<p>And why? Why indeed. Why is online education simulating University education when it actually has no relationship to it? You aren't getting a University qualification by completing a "class," there is no reason why a "class" has to be X number of weeks or you should have to complete the rereq's in order to take it.<p>Places like Khan Academy have got this right. I can go to Khan Academy right now, click on a video series, watch a few bites of information and then stop when I'm ready/bored. Coursera's class system is just pointless, stressful, and annoying.<p>If Coursera was transferable then it would absolutely make sense to do. But that doesn't look likely and while that isn't the case they're just making access to learning material more difficult with no obvious benefit to anyone.
评论 #5472134 未加载
评论 #5470702 未加载
评论 #5473615 未加载
DennisPabout 12 years ago
I actually often "never really intended to seriously take the class in the first place." Coursera doesn't let you view the materials unless you sign up for the class, so I sign up for every class I might be interested in browsing a little. Then I can view the materials at leisure, even long after the class is finished.<p>On the other hand, I mostly completed the first algorithms course and Model Thinking, and while Daphne's class kicked my ass the first time, I'm already reviewing to tackle it again and try to finish it this time.<p>It is true though that the time commitment is horrendous, especially for Daphne's class, which estimates 15-20 hours per week! If I fall behind, I may finish slower than their schedule and miss the homeworks and final exam, in which case I'll officially be an incomplete but I'll still have gotten through all the material.<p>In short, retention is an irrelevant metric for online classes, especially for Coursera which has such a strong incentive to sign up "unseriously." My suggestion would be to keep accepting and grading homeworks and exams at any time, and give extra props on the certs for completing on time with the other students.
Jormundirabout 12 years ago
You should title this article "How to cheapen the online education experience". You may be right, that edX and Coursera require a big commitment, but that's what learning requires. Do you want to play games for an hour, or do you want to advance your skills and understanding?<p>(I've completed the Codeacademy Javascript track, and a full Coursera class (livin' in the 5%, wooo!)).<p>You say Codeacademy got it right, but I think you're dead wrong. As other articles today have pointed out, at best you're going to learn syntax on codeacademy, and the most basic programming principles, but you'll be miles away from being a capable programmer. The jump from the online editor with tiny little exercises to setting up your own environment and programming your own project is huge. Many of their lessons give misleading or outright incorrect information, not to mention teaching you bad practices.<p>What this article says is the huge barrier to entry, is not a barrier to entry to online education, but a barrier to entry of learning. What the author proposes is solutions to hide the barrier of learning, by gamifying the platforms and making them more "fun" with a disregard for the depth of effectiveness, and the consequences of pursuing such techniques. (Obviously I'm delving into opinion land, but I strongly believe gamification cheapens and platform and doesn't produce the deep engagement necessary for learning. This is my assessment from my experiences, I'm not going to tell someone their wrong for feeling differently about this. But I think if you wanted to create a deeper experience from gamifying education, it would take a tremendous effort, far different from the typical "Wow you completed 5 exercises! ZOMG KEEP GOING!" joke of rewards that other websites have.<p>I don't want to rail on the online education effort either, but this is plain cheap thinking. Coursera and EdX are doing an excellent job, but they certainly have many problems to surmount. They've definitely solved the problem of higher education accessibility, now the problems they have are in the effectiveness of education, and on administration and grading that gets the f#$% out of the way of learning. But making it "easier to learn"? I call utter bulls$%t.
评论 #5471289 未加载
nnqabout 12 years ago
Why do people think <i>retention</i> is a good thing?! I think <i>retention</i> is <i>actually bad</i> even for classical education.<p>The lower the retention rate, the more the likelihood that they are following lots of courses at the same time or multitasking with something else. This means that <i>people are "exploring" more!</i> I think that "exploratory learning" has always been stifled by classical organized education and is one of the reasons why I hate most academic environments (yeah, they're cool if you're in one of the top 10% unis or in a "privileged" position", but not for the rest of cases...). I believe that, after a certain level of baseline knowledge, in any field, it's actually <i>more important that someone learns "what they want/need/have inclination for/find more interesting" than that they "learn more"!</i> Maybe more time exploring and less time actually uploading things to your mind is better (not "productive", just "better", and I said "fuck productivity" a lot lately because I found that it just doesn't lead to better anything). All the new ways of doing education make exploration easier (as in you can explore a lot without really "wasting" that much time), and we should take advantage of this!
darrellsilverabout 12 years ago
Pacing is incredibly important to motivation. The thing is, there's no single answer. Even in traditional, full-time, in person classrooms where everyone puts in the same class hours, natural ability, different skills at the beginning and better attention mean that students will always learn at different speeds.<p>The ideal model is one of personalized education. What's kind of shocking is that online education so far has basically ignored this altogether. There's been zero investment in ways to continually pace and encourage students to learn at a speed that is both challenging and within reason.<p>This isn't too surprising: it's a hard problem, and it's much easier to have users paying a subscription fee forever (for those services that are paid) and blame themselves when they fall behind.<p>At Thinkful (<a href="http://www.thinkful.com/" rel="nofollow">http://www.thinkful.com/</a>) we see evidence that our learning model, which pairs experts and students together much more like you'd expect from tutoring, sees 10x the retention and completion as other online course options.
评论 #5472151 未加载
KC8ZKFabout 12 years ago
My mother used to tell me that my eyes were bigger than my stomach when I took food more food than I could eat. I have the same problem with MOOCs.<p>The future me always has more time, is always more dedicated to learning, is always more focused. So I sign up, and when the class starts I'm no longer the better, stronger, faster, future, me, I'm just me.<p>Still I manage to finish some, and learn a little from the ones that I drop, so I don't see it as a problem.
droithommeabout 12 years ago
Another week, another article complaining that teaching a class which 5-10 thousand people successfully complete, and which cost less than $10 per successful student to provide, is an abysmal failure.<p>Meanwhile, the schools that the authors hold up as more successful examples of education have 20-500 successful graduates per session offered, nearly all of whom are idle young wealthy white people living in rich western countries, and who pay up to several thousand dollars per session each for this privilege.
jeffdavisabout 12 years ago
The article seems to imply that, in an ideal world, all of these student would stay throughout the course. But I don't think it's clear that the churn is bad.<p>There is some lower bound to the amount of time that it takes to learn something (absent some educational revolution). Many people aren't even willing to pay that lower bound simply because they have other priorities in life. But they do like to sample (which is good), and potentially finish out a course if nothing else gets in the way.<p>Online courses are great because they lower the cost of sampling, so we shouldn't be surprised that there are more samplers, and fewer people finishing out the courses. If 1000 people complete an online course, then that's great, even if 99000 people signed up and disappeared a week later.<p>That being said, I'm pleased to see the specific criticisms offered in the article, and I hope they lead to a better balance for more people.
waterlesscloudabout 12 years ago
The author says courses shouldn't have deadlines, but then talks about retention that's measured as of the deadlines.<p>I signed up for the Hinton neural network class when it was almost over, and completed it well after it ended.<p>Was I "retained"? Did I not count as completing the class?<p>Just because there's deadlines doesn't mean you have to finish it by the deadline.
kvbabout 12 years ago
Thankfully the article is much better than its title. Low retention is not a problem <i>per se</i> - since there's no cost to signing up for (and subsequently dropping) these online courses, it's to be expected that retention won't be great. However, I agree that retention could probably be improved among the marginal subscribers by using some of the techniques covered in the post.
tunesmithabout 12 years ago
For the past year, I've only been seriously interested in three Coursera courses - Scala, Ng's Machine Learning, and Probabilistic Graphical Networks.<p>All three overlap at the same time!! And they don't give assurances of when they will be offered again, or even if. Argh. I've started Scala but doubt I'll be able to continue if it I attempt PGN again. This happened to me last fall and I ended up completing none of the three.<p>Plus, it is apparently "known"... somewhere... that certain courses are easier if you take other courses first... but good luck finding that information when you want to refer to it.
评论 #5471209 未加载
andrewemabout 12 years ago
My wife also wanted to take that Introductory Human Physiology course. She's a freelance medical translator, so improving her knowledge of that subject matter will have a direct positive impact on her career. She's not among the people who Daphne Koller said "never really intended to seriously take the class in the first place" - she bought the suggested books ahead of time, marked it all down on her calendar, and planned out time to work on it.<p>What she found was that the course was way more difficult than the description suggested, and proved to be well above what she could understand with her current background. She watched the first video and found that it was going to take way more time than the course description predicted, because she would have to try to fill in a tremendous amount of background knowledge that the teachers presumed. On the plus side, from the course forums she found about resources (videos, etc.) that were more at her current level, and has been working through these.<p>The comments made elsewhere about the synchronous model for these courses is probably right for her case - she had to wait a number of weeks in order to find out it wasn't the right class for her. This is what you expect when you're in college, at which point you either drop the class and maybe find a different one to take, or tough it out in a class that's not as interesting or useful as you'd hoped. I hope online courses can do better than this.
评论 #5470305 未加载
denzil_correaabout 12 years ago
One of the main problems the author has with Coursera or EdX is that it is "too fast". To address this problem, I urge the author to try Udacity [0]. You can set learning at your own pace.<p>That said, not all material on Coursera is "amazing". Some of the classes have very high completion rate like <i>Functional Programming with Scala</i> has a 19.2% completion rate [1]. Similarly, the class taken by Andrew Ng on Machine Learning is fantastic. However, many of my peers had bad reviews on Daphne Koller's Probabilistic Graphical Models class. Last year, I myself registered for one of these MOOC's and found that half of the course was good while the other half was quite bad - both halves had different professors.<p>At some point of time, universities would have to realize that great researchers do not make great teachers. Some excel in both - researching &#38; teaching while some in just one of those two fields.<p>PS - Other problems on reengagement do stand though.<p>[0] <a href="https://www.udacity.com/" rel="nofollow">https://www.udacity.com/</a> [1] <a href="http://www.katyjordan.com/MOOCproject.html" rel="nofollow">http://www.katyjordan.com/MOOCproject.html</a>
评论 #5470955 未加载
michaelfeathersabout 12 years ago
<i>We could come up with a bunch of excuses about why online education will always have worse retention, or we could figure out ways to fix it.</i><p>Or we could ask whether it is a problem. I'm not sure that it is.<p>I don't think the numbers are surprising or unnatural at all. The thing to compare them against is not traditional education but rather the patterns of all people who take the autodidactic route.. every person who has ever picked up a book and not finished it. Even then, the comparison isn't perfect.<p>In fact, we could argue that low retention numbers are actually a good thing for non-credit courses. It means that the content is challenging and focused, and that people are able to discover early whether it is or isn't for them or cherry-pick the content that they need and move on.<p>If there are usability problems, fix them. But if participant retention got dramatically higher, I'd worry.
noliteabout 12 years ago
really great points in this article. I hope that Coursera sees this and really takes it seriously. I love what they're trying to do, and have signed up for about 30 classes myself that I totally intended to study, and just couldn't finish a single one. And its true, artificially imposing the traditional college course model just does NOT make sense in the online context. Why are there deadline for problem sets, and test schedules? Fix that, and it'll change everything
fcorrabout 12 years ago
I'm taking two Coursera courses at the moment - Calculus One and Pattern-Oriented Software Architectures for Concurrent and Networked Software. I'm very pleased with the material and presentation in both, but I don't think I could handle more than two such (relatively difficult) courses at any one time.<p>Codecademy and Udacity allow you to freely dip in and out of lectures and assessments, whereas Coursera courses demand that you deliver quizzes/assignments every week. It's hard to recommend one approach over the other -- with weekly deadlines I find I have more of an incentive to engage in the lectures and course materials, but the added failure conditions can push students to abandon the course.
johnwatson11218about 12 years ago
One of the things to that I found confusing with the coursera classes was that each one seems to have its own set of rules for dealing with homework assignments, late homework, and partial credit. At one point I was signed up for two classes at the same time and I assumed they had the same policy for when homework was due. It wasn't a big deal but there were several times I would have a free hour or so and it was a task to figure out the best way to use that time. I think it is a positive thing that the courses are trying out a lot of different approaches to see what works best but this reminded me of the parts of academia that I didn't really care for.
steveinflowabout 12 years ago
I don't think low retention is a secret and I don't think it's a bad thing- it shows that there's no barrier to showing up but there is a barrier to completion. That's what we want- equal opportunity, without giving completion away unearned.<p>For my part I sign up for more classes than I could possibly complete. Sometimes I just want to take a peak at the material to see if it grabs me. And sometimes I just want to get familiar with a subject without going all-in, picking and choosing the lectures that are most relevant to me.<p>People shouldn't feel like they're hurting Coursera's/EdX's/Udacity's stats by dipping their toes.
jurassicabout 12 years ago
There will continue to be low retention until there is an incentive to finish. Without the credentialing piece, I don't see a problem with low retention. I think it's awesome that people dip in and out as their needs and interests change. In every class I ever took at MIT/Caltech there were at least a couple lectures I would have been happy to skip. MOOCs enable that.<p>I hope there will be more short and tightly focused online courses in the future instead of monolithic traditional-length courses we have today. I think that'd go a long way to improving the apparent retention rate.
graycatabout 12 years ago
I believe on-line learning can work.<p>Why? Three reasons:<p>First, a 'dirty little secret' of the US software industry is how much of the learning from the beginning of electronic computing to the present has been just from individuals teaching themselves from books, e.g., K&#38;R on C, Lippman on C++, Ullman on database, Sedgewick on algorithms, and on-line, e.g., Microsoft's MSDN site, StackOverflow, etc., essentially independently without courses, lectures, problem sessions, credits, homework, tests, etc.<p>Or since just K&#38;R, ..., StackOverflow, etc. have been responsible for so much learning so far, then 'the bar is low' and on-line courses should do even better.<p>Second, in my experience in technical subjects, pure and applied math, mathematical physics, some topics in electronic engineering, e.g., surrounding the fast Fourier transform (FFT), and software, with a class or not, nearly all the learning (in my case) took place from study, alone, in a quiet room, from good materials just on paper. My 'educated guess' is that on-line learning can't replace such learning but can help stimulate more of it.<p>Third, for a researcher in applied math and software, and also likely some other fields, one of the main 'work items' is to take recent books and papers and work through them much as working through advanced course materials in such subjects.<p>Back to my case, in the fields I worked hardest on and did the best in, math, physics, and software, starting in the ninth grade, through my Ph.D., and in my career to the present, I did nearly all the work with relatively little contribution from teachers. E.g., in plane geometry my teacher was the most offensive person I've ever known in education, and I slept in her class and refused to admit doing her assigned homework. Instead, I worked every non-trivial problem in the book including the more advanced supplementary problems in the back where she never made any assignments. Then, after working all those problems, no wonder, on the state test in the subject, I did fine: I came in second best in the class; the guy who beat me also beat me by a few points on the Math SAT -- we were 1, 2 in the school. Net, my approach to learning plane geometry worked fine.<p>I never took freshman calculus. The college I started at wanted me to take some math that really was just a review of what I'd covered in four years of math in high school. So, a girl in the class let me know when the tests were, and I showed for those. The teacher said I was the best math student he'd ever had. Meanwhile, I got a good calculus book and started in, worked hard, and did well. For my sophomore year I went to a much better college and started on their sophomore calculus and did fine. Yup, never took freshman calculus.<p>When I went to graduate school, I took a problem with me and had an intuitive solution. My first year had some good courses (one was just terrific, from a star student of E. Cinlar now long at Princeton) and gave me what I needed to turn my intuitive solution into a solid math solution; I did that in my first summer, independently; and that was the research for my Ph.D.<p>I continued that way in my career: E.g., in a software house working for the US DoD, I saw a problem in a specification, got Blackman and Tukey, 'The Measurement of Power Spectra', and read it carefully enough to see what was wrong with the specification and how to fix it. Right: without courses, lectures, problem sessions, ....<p>As far as I can tell, nearly all the technical content on HN, StackOverflow and other Internet fora is from people who taught themselves in similar ways with little or nothing in courses, lectures, problem sessions, .... And that's part of what researchers have to do and is just part of getting tenure as a research professor.<p>So, since so much work is being done by essentially independent study now, just by not making things worse on-line courses should be able to look successful.<p>But I see some problems with the on-line materials I tried:<p>(1) The video quality just sucked. I couldn't read the board. That meant I couldn't copy what was on the board and study it. Bummer.<p>(2) The sound quality was not good enough.<p>(3) The course materials, e.g., on paper or in PDF files, were from not good enough down to just missing.<p>(4) Sadly the quality of the course content was too low; apparently the main reason was the desire to make the course more 'appropriate', that is, 'easier', for 'the common man in the street'. But, omitting material 'waters down' the course content and, really, for a good student, requires that they fill in the gaps for themselves -- bummer.<p>E.g., I looked at the course by Stanford professor Ng on 'Machine Learning'. What I saw were weaknesses (1)-(4) above. For more, (A) a lot that he was doing was maximum likelihood estimation but with far too little explanation and justification; so, I would have had to have run off and studied maximum likelihood estimation on my own. So, again I was on my own to do some independent work, trusting Professor Ng that somehow maximum likelihood estimation was better than it has long seemed in the statistics community. (B) He mentioned the 'maximization' to be done via following gradients, and that is an overly simplistic and not very promising approach to maximization -- the standard, first problem is that spend nearly all the computing time moving in directions nearly orthogonal to the direction really should be moving in.<p>So, from (A) and (B), I concluded that for a good course I would have had to have taken his lectures just as topics to be investigated, gone to good materials elsewhere, collected good details, and written my own text. That's his job as a professor, not my job as a student. His field, 'machine learning', didn't look worth that much work for me now. I've done some serious work in several cases of applied math that could be called 'machine learning' as much as his material, and I'm left without much respect for his material.<p>I looked at the course 'Probabilistic Graphical Models' taught by Daphne Koller. Since I very much liked a course by a star student of E. Cinlar, maybe I should like Kollar's course. Sorry, I didn't -- the quality looked too low. Better quality from Stanford? Sure, K. Chung, H. Royden, D. Luenberger (his 'Optimization by Vector Space Methods' is a beautifully, even elegantly, done one mile long applied math dessert buffet), D. Knuth.<p>For courses in 'how to code', that is, introductory material in software, gotta be kidding! 'Coding' alone is easy; it's just, pick a language, learn the basic syntax, and write if-the-else, do-while, call-return, allocate-free, etc. It's easy but doesn't take one very far. So, don't get very far with thousands of Web pages of documentation at MSDN on .NET, ASP.NET, ADO.NET, administration of SQL Server, IIS, Windows Server, etc. or the equivalent in the Linux world.<p>I have two broad conclusions:<p>First, traditionally in academic material in technical subjects, the author and the student 'reached' to each other, and they connected at a well written textbook. So, the author went far enough toward the student to prepare a good text -- and the best texts are terrific. And the student reached far enough toward the author to make do with little or no more than a good text. It's how I learned plane geometry, freshman calculus, theoretical, applied, and numerical linear algebra, everything I learned about statistics, most of what I learned about advanced calculus, signal processing and the FFT, stochastic optimal control, artificial intelligence, ..., and everything I learned about software. E.g., it's heavily what worked for me.<p>The on-line community will have to face the fact of this 'reaching'. In effect, on-line learning requires the professor to do more work of a kind that promises not to be well rewarded by tenure and promotion committees. So, net, so far a student should still reach mostly for one of the best texts, on paper or PDF.<p>Second, the situation will 'settle out': The professors will come to understand what minimum quality is needed, and the students will realize that the learning is not just a spectator sport, is not like watching a movie, and still requires nearly all the traditional work from a book or PDF file. Then the courses will get better; the students just looking to watch a movie won't sign up; and the course completion rates will increase.
评论 #5470209 未加载
tsikiabout 12 years ago
I agree with the sentiment, but I wonder if having a low barrier of entry is really the solution, or does having a low barrier of entry average out to a low level of initial commitment. It'd be interesting to see if an online course for which you pay for would be better at retention. From a business point of view, they could even offer to give you the money back if you finish the course, or use some other retention trick, like have people calling you to create some social pressure if you seem to be falling behind.
评论 #5469876 未加载
评论 #5470055 未加载
评论 #5469751 未加载
tokenadultabout 12 years ago
I've bookmarked this interesting analysis to add the link to my personal website. I especially like the comparisons among differing brands of online courses.<p>Before I read the fine article (which I recommend you read too), I was going to react just to the headline. And that reaction is still there. The reason we (correctly) criticize lack of retention in online education is that online education makes apparent metrics that show the lack of retention. But I would suggest, for learned discussion by all the participants here, that most forms of classroom learning, and especially compulsory secondary schooling in most of the developed world, suffers from a very bad lack of engagement on the part of the students. (Some of the comments posted here before this comment tell stories about that.) A high school student may attend classes often enough to avoid being expelled and gain a diploma, but the student may operate in "regurgitate and forget" mode the whole time he is in high school, not really remembering anything he supposedly was taught. The one kind of deep learning many high school students engage in is learning how to pretend to be studying. If online education develops better demonstrations of learner acquisition of knowledge and skill than mere seat time in a classroom, it may be a powerful force for exposing the dirty secret of classroom education.
zallarakabout 12 years ago
The title of this article is very exaggerated and needlessly so. The only reason you see awful retention numbers is because it's so easy to sign up. Even if I'm only partially interesting in a class, I'll sign up. I often sign up for 4 or 5 classes that I'm interested in and will finish 1 or 2 per term, just so I don't miss the chance to sign up later/have access to course materials.<p>If <i>you</i> have issues with the class, that's a personal problem and you simply need to work on your willpower.
hojoff79about 12 years ago
I think the main issue is a matter of motiviation for the classes. And I do not mean people being lazy, but the original impetus behind signing up for these classes.<p>For the Coursera classes I did sign up for, I did it because I am interested in the subject area, know almost nothing currently and wanted to gain some functional understanding. I would say of the classes I signed up for, and probably 50% of those I have seen all the lectures. But then I go off and incorporate into side projects to test out / make sure I understand the materials. I have never turned in an assignment, (almost) never posted on a message board and never opened a final exam. Why should I? I am going straight to the application of the content into my projects, I've already achieved my goal of understanding and expanding my capabilities.<p>What reasons would someone have for wanting official completion of the course? On most courses, you do not gain any officially recognized credit, the certificate has limited (if any) commercial marketability for someone... so again, why would I want to go through the hassle of officially completing the course?<p>Two takeaways from these thoughts are the following: 1) How many people like myself have utilized the content of the course but are not seen as retained due to the fact they do not partake in the academic completion side? 2) What potential motivators can exist in the future to drive completion of the academic side of the coures? (college credit is the first one that comes to mind, but still does not help motivating many users who are above that age). What other drivers could come into being to incentivize?
kvasanabout 12 years ago
Things work differently for different minds, I find that Courseras model of commitment works better for me. I tried 2 Udacity classes which are self-paced, I actually did quite a lot of the modules but somehow I moved on. On Coursera you have to live with the class and at least try to make all assigments and such on time. I think one thing that works good for me is the feeling of having a special oportuinty and "if i dont do this I will be left behind in the new age". Dont get me wrong I would love to be the kind of person I thought I was; searching the web, finding places and things most people would not assume were there and then putting them together. But I need the feeling of being on a "quest" and building up a illusion of "if I make this, things will change". Latter rambling is maybe more general, my point is some people (me) just cant handle the notion of almost force-feed overflow of accesible information and thinking that everybody doing it, so why bother (if the retention in the article is true that should be instrumentaly good for my crazy notion). Sorry for the crazy rant, if only the mind lacked biases. Maybe I should atleast try those mentioned in the article instead of being overpowered by strange thoughts.
johnminterabout 12 years ago
The hard truth is that the student gets out of a class what they put into it. I have finished 3 Coursera classes, two of which gave certificates. All three were well worth my time, but I worked my tail off to complete the challenging assignments. I found the support staff (TAs) helpful in the forums. Life does get in the way some times but all three coures had at least a week to finish an assignment, typically two and the option of applying late days on quizzes.<p>The biggest thing I noticed were the clueless forum posts by students who obviously had not read much of the provided material. Two of the classes had material similar to ESR's "How to ask questions the smart way" and this was largely ignored. I think we are seeing the fall out of the distraction of modern society. Sadly, tl;dr doesn't work with complex technical material.<p>That said, it was a privilege to interact with motivated fellow students. Their input in the forums helped eliminate misconceptions.<p>By the way, the statistics for our Data Analysis class were similar to those from the OP - from the instructor: "There were approximately 102,000 students enrolled in the course, about 51,000 watched videos, 20,000 did quizzes, and 5,500 did/graded the data analysis assignments."
eah13about 12 years ago
tl;dr: What's at stake here is the proper roles of ed tech in education. MOOCs are media, only one part of the ecosystem that is developing here, and I think that finding a central role for great educators is key.<p>Coursera and other MOOCs are media properties that monetize viewership (thorugh certificates rather than ads). As media properties, michaelochurch and others are right that the synchronous model isn't really going to work in the long term.<p>I think ed tech should be about connecting teachers with the content they need to teach (and I'd include self-educators amongst 'teachers' here). Students need synchronous access to teachers, but those teachers don't necessarily have to have created that content just now. This suggests that online education might look more like a 1:100 teacher-student ratio rather than 1:100k. Nothing's more motivating and engaging than a human being.<p>Personally, my best experiences have come from two sources: self-study from great materials and guided study from great teachers. This suggests that a role for ed tech might be connecting great teacher with great content rather than broadcasting teacher-as-content. Personalization, engagement, motivation, etc all come for free if we empower great teachers rather than seek to replace them.<p>It's misguided to think that some professor from a great school can best deliver the content to all audiences. Imagine great open content accessible to every teacher for use in their own classrooms. With an inverted model of teaching, the lets them push content out to the students for self-study and then have synchronous classes where they, the teacher, can motivate and customize the delivery of tough concepts for their particular group of students.
评论 #5472990 未加载
mani27about 12 years ago
Even if 5% people(say 5000 people) complete the course, whats wrong in that ? 5000 is still a big number. Many Prof's wouldn't have taught that many students all their life. I have myself completed 7 courses from edX,udacity and coursera. Each of these have a different approach. Udacity for beginers while edX and coursera have some advance courses and I found these courses a great supplement to my course work at my school.
评论 #5469947 未加载
walrusabout 12 years ago
I'm more likely to finish a few chapters of a textbook than a series of videos:<p>1. Videos are usually too long. There isn't a natural break to stop and try things out. (Udacity does a good job with this one.)<p>2. Information density isn't constant. With text, this isn't an issue: I naturally adjust my reading speed if I hit a sparse or dense area. With video, I have to either live with it or constantly re-adjust the video speed.
zabramowabout 12 years ago
I'm a big fan of Codeacademy myself. But, as someone who doesn't come from a technical background, I feel like I'm often completing many of their lessons, but not internalizing the information or actually learning how to code.<p>I don't plan on giving up, but I'm curious if others have had that experience or if I'm the only person who's figured out how to use Codeacademy without actually learning to code.
teabout 12 years ago
MOOC providers could enhance engagement and retention by providing audio-only content in addition to or even instead of video-based content. I've got a couple hours per DAY where I crave quality audio content but am physically unable to watch videos at that time. In contrast, I've only got a couple hours per WEEK that I can watch videos.
评论 #5470322 未加载
lessnonymousabout 12 years ago
I find most of the comments on this post fascinating. Probably partly due to the audience, and partly due to the slant the author took on the topic.<p>His bait is retention .. and that's something HN readers understand from a business POV. But retention isn't his issue. Retention is a problem for Coursera et al.<p>His problem (and mine) is that we (personally) found it hard to stay the course. And Coursera don't help us.<p>I've a full time job, two side projects, a small business and a family. I'm about to start the Gamification course and I probably wont finish it because I don't have time. Not because I don't want to, but because the course expects all students to spend 4-6 hours per week studying. Those weeks would be rare for me.<p>How hard would it be to offer a 2 hour-per-week version? Or whatever commitment I'm able to make? Then, when the inevitable happens, let me hit the 'pause' button.
mark_l_watsonabout 12 years ago
That was a well thought out article, a good read, and I don't agree with most of it. I think Cousera is Amazing.<p>The only thing I don't particularly like is the inflexibility on turning in assignments. I completed five classes but did not pass one because something happened work wise that caused me to turn it several assignments late and that really knocked down my score. Great class though.<p>I have also started four other classes that I dropped after 1-3 weeks. I am still glad I was exposed to some of the material for those classes, so still a positive experience.<p>Accepting the figure that 95% of students drop classes on average, I would like to know if most of those people in general felt their time had been worthwhile spent. That is the important question.
jimbobimboabout 12 years ago
I did one Coursera course, where I'd finish Q&#38;A tests. But the course I'm taking now requires me to do some video/audio assignment, share it with peers and provide peer critique. Sorry, but I have a life, so I'm there just for the content.
auctiontheoryabout 12 years ago
Coursera courses are supposed to be college courses. College courses are a lot of work, and you have to get your reading and assignments done on time. This should not come as a surprise.<p>CodeAcademy (also wonderful) is a completely different animal. It is not comparable to Coursera.<p>What you're calling a bug (aka "dirty secret") of Coursera is actually a huge feature: that we can all, whatever our goals or our budget, sign up and check it out. Quite unlike an class at MIT or Duke, which is only available to a pre-selected few.<p>In short, the MOOCs, and specifically Coursera, are awesome. We are so lucky to have this opportunity. Quit yer bellyachin'.
vdpabout 12 years ago
A relevant graph: <a href="http://www.katyjordan.com/MOOCproject.html" rel="nofollow">http://www.katyjordan.com/MOOCproject.html</a><p>IMO for a class with tens of thousands people signed up, 5% retention rate is not really that bad.
tanseyabout 12 years ago
I saw a talk from Andrew Ng a couple of weeks ago where he was asked about the low retention rate. He countered that 40% of people who submit the first homework actually complete the class. I would say that is pretty great retention rate.<p>Personally, I've signed up for dozens of Coursera courses but I've only completed one (Game Theory). It's not necessarily that the other courses were lacking the ability to keep me enrolled, but rather I never planned to try at them. I sign up to most classes to see the videos and get surface-level understanding of a subject.
timedoctorabout 12 years ago
I think a major problem with retention is that the course is not a recognised qualification.<p>The next step for online education is get a real qualification that improves your job prospects through places like Coursera.<p>Also I think there are a bunch of other hacks such as collaborative motivation (getting a group of members to commit to each other to complete the course and automatically reporting to each other when they have completed each section where everyone in the group has collective responsibility to help each person in the group to complete the course.
sefkabout 12 years ago
I don't necessarily think this is a bad thing.<p>For Stanford's database class that just finished we had 64 thousand register, but only 20,000 actually do some work, and then close to 5k actually take the final or get a statement of accomplishment.<p>My blog response: <a href="http://sef.kloninger.com/2013/03/online-ed-retention/" rel="nofollow">http://sef.kloninger.com/2013/03/online-ed-retention/</a><p>I'm the engineering manager on Class2Go, Stanford's open-source MOOC platform. Check it out: <a href="http://class2go.stanford.edu/" rel="nofollow">http://class2go.stanford.edu/</a>
评论 #5471878 未加载
jiggy2011about 12 years ago
Does it matter if not everyone watches right until the end and completes the final coursework?<p>I want to know as much as I can about different subjects but I want to immerse myself in them to different levels.<p>Sometimes I am happy just to stick a few videos on in the background while I do something else, so I get the gist of what something is about and know what to focus on later if I need more information.<p>I don't necessarily care if I could pass an exam or not.
hkonabout 12 years ago
An anecdote, I followed this course this year, <a href="https://www.coursera.org/course/growtogreatness" rel="nofollow">https://www.coursera.org/course/growtogreatness</a> and the number of student increased every week.<p>The guy presenting, Ed Hess, was very skilled and interesting and the material was very relevant. I also enjoyed the discussion of various homework on the course forum.
futheyabout 12 years ago
Since the barrier to entry for enrollment is zero, compared to traditional education, directly comparing them is rather unfair.
rigginsabout 12 years ago
I'm curious about enrollment trends.<p>Having taken online courses from the start my sense is that enrollment/participation is slowing. The bulletin boards of some classes are pretty dead on the first run.<p>Its not crazy to think that the people most likely to take the classes would participate right away and finding the next batch of students will be harder.
demianabout 12 years ago
"Awful" is kind of relative.<p>We are talking about a new model, low "retention" seems to be normal for free clases wich have absolutely no barrier of entry.<p>I have taken 5 courses, only finished 2 "on time". For the other 3 I downloaded the material, got some exercises on sites like MIT OCW, and finished them at my own pace.
sopooneoabout 12 years ago
At a fitness center, personal trainers serve the role not just of instructing their clients, but of renewing motivation. That task is absolutely dependent of physical proximity. I feel like this dynamic is being completely ignored in most discussions of online education.
gfodorabout 12 years ago
Learning is hard. Self-study is very hard. Incentives matter.<p>The main incentive to power through a Coursera course is personal enrichment. This is not the reason most people go to college.
donniezazenabout 12 years ago
Online education has no similar incentive as formal education has. And one of the most important aspect of online education for me is to learn what I find interesting.
jcmontalbanoabout 12 years ago
This article exposed me to a novel usage of "churn".<p><a href="http://en.wikipedia.org/wiki/Churn_rate" rel="nofollow">http://en.wikipedia.org/wiki/Churn_rate</a>
laughfactoryabout 12 years ago
Just to throw my hat (opinion) into the ring...<p>I've tried pretty much all of the "MOOC's" out there, and so far the only two I've stuck with were Codecademy and Udacity. Why? Precisely for the reasons that are mentioned at the article which started this discussion. The format of Edx and Coursera is just all wrong for most people who DO want to learn something but have a busy life that is not conducive to a traditional college class.<p>Besides which, all too often, Coursera and Edx just throw a college class online (lectures, course notes, homeworks, quizzes, and tests) and call it a day. Then they have arbitrary deadlines--which I acknowledge might help some people but the first time I got a deadline email I decided that it wasn't for me...no matter how interested I was in the class topic. I'm already married and a father who is working part-time while doing a Master's program in Computer Science. So I don't have huge chunks of time available for traditional course work (in addition to my Master's coursework). I have little bits of time every day which I use effectively to progress through the Codecademy and Udacity coursework. Slowly but steadily I'm getting through both and making substantial progress.<p>But, since Coursera and Edx require huge blocks of time committed every week, and a tight timeline to complete the course, I just don't find it's workable. Which is sad since they seem to offer some tremendous material.<p>In short, I think that Coursera and Edx (and probably colleges in general) should learn a thing or two from the highly modularized approach of Udacity and Codecademy.<p>I find that my learning retention and interest remains much higher with both Udacity and Codecademy because they present a little material (little bite-size chunks) which slowly build into more and more challenging material and intersperse "challenges" or "quizzes" throughout--rather than the traditional approach which--by necessity--was far less responsive.<p>The fact is that online learning doesn't just reach potentially numerous people that wouldn't have otherwise been exposed to such educational opportunities at such a low price (currently free), it potentially allows for a radical rethinking of the traditional education model.<p>I think that it is sheer ignorance to say, as some have in the comments I've read (and as--apparently--Coursera maintains), that those who don't finish a course just aren't interested enough or motivated enough. This is ridiculous. The very same (allegedly "unmotivated" or "undisciplined" or "uninterested") people may be successfully working their way through Codecademy or Udacity. And yes, it may take them a while, but eventually they'll complete a Codecademy "track" or a Udacity course. And they'll do it one bite-size chunk at a time...and sometimes, say on a weekend, they'll plow through a whole bunch of bite-sized chunks.<p>When Coursera and Edx adopt a more bite-sized format which isn't oriented around arbitrary deadlines, I'll check them out again. Until then I'll stick with Codeacademy and Udacity.<p>Bottom-line: the standard model is broken. Lectures and homeworks and quizzes and tests are poor ways to educate. For many years they were the best we could do within the technological constraints we faced. But now we can do better. Heck, I'd encourage everyone working the bleeding edge of online education to think "outside the box" and try new approaches. While the best new approaches I've seen have been those utilized by Udacity and Codecademy, that's not to say that there aren't even better methods--methods better at retaining learners (keeping learners engaged), AND methods which excel at increasing retention of material.
newobjabout 12 years ago
Seems a bit early to optimize for retention. Or to have dirty secrets. Or real suffer almost any kind of analysis at all.
mikecaneabout 12 years ago
I've not tried these. Do they all require you to sit at a desktop PC or are they tablet-friendly or tablet-customized?
Turing_Machineabout 12 years ago
Another article that equates "MOOC" with "online education".<p>Sigh.
tsumniaabout 12 years ago
Retention rates are low in everything, not just online classes. Ultimately, its how the student reacts to the plateau. Anyone in any area of teaching, I'd highly recommend reading "Mastery" by George Leonard. In the book, he talks about 3 of learning types that can stifle progress: the Dabbler, the Obsessive, and the Hacker (not to be confused with the IT world's rough definition).<p>* The Dabbler will pick up tons of things, "overjoyed" about the newness that they are learning. Eventually, they hit their first plateau and the 'newness' becomes boring and they'll seek out another activity to experience the same 'newness' high. Maybe they'll come back to it later and see if they can progress again.<p>* The Obsessive will seek out perfection on this new activity, pouring over every YouTube video or article to figure out how to be better. When they plateau, they'll try harder; hoping for the same results. If the return isn't there, the Obsessive will look for another activity.<p>* The Hacker will actually ride out the plateau potentially indefinitely. In some cases, this will create progress eventually, but in others, it can become the cap of their learning capability (hence, 'a hack'). They just don't put the extra effort needed to progress further.<p>While all these types are negative, two-thirds drop the activity for something new, something they might be a natural at. From a business perspective, a monthly subscription favors Hacker-type customers; where a 'per course' pricing scheme favors Dabblers.<p>Steering back to online courses, I think the issue isn't 100% the site's fault; but that certain students will join for the excitement, but fan out eventually (Leonard goes on later to discuss how to better approach each of these types as the Teacher). I've signed up to all of the sites in the article, along with Udacity (where I'm currently taking the Web Dev course). For Udacity and edX, I've completed the courses I've signed up for; however at Duolingo, I stopped after the Basics 1 in Spanish. I want to learn Spanish, but the drive and time isn't there for me.<p>Jumping back to 'Mastery', Leonard talks about 5 keys that can help promote mastery and can be used to retain students: Instruction, Practice, Surrender, Intentionality, and The Edge.<p>* Instruction is obvious: have great teachers(not just through credentials, but are generally good AT teaching), something these sites offer tenfold.<p>* Practice is where I felt edX's course fell through. It was a lot of lecture, but the Finger Exercises were more the tradition 'regurgitation' of knowledge from traditional classes. Duolingo and Udacity (at least the Web course) seem to have this concept taken care of. Duolingo offers different ways to practice and at least with Web Development, you can to see the fruits of your labor with each class.<p>* Surrender is more for the student than the instructor, but talks about accepting you aren't a Master as something and so sometimes you need to take two steps back to take one step forward (in Aikido, we refer to this a 'Kihon Waza', or practicing the basics). So many people have the mindset of "If you're not doing it this way, you're doing it the wrong way", that this can stifle learning. As an instructor, thinking up new and creative ways to practice the basics can help alleviate this.<p>* Intentionality talks about visualizing the action, and while focused more on physical activities, could be utilized online. Give the students a view at the end results/expected return on their time. Sure, its dangling the proverbial 'carrot on a stick' in front of them, but it can help them sees where the fruits of their labor. If you can mix this with some of the Practice key, you could get better results (IMO).<p>* The Edge tip-toes the line between endless practice and pushing the envelope (though Leonard says it isn't). For a new student, this can be a little harder, considering they are still learning; but creation of 'outside the box' milestone/projects could benefit from this. The student's amassed this bit of knowledge; make them use it in a less than cookie cutter way. This would be a great undertaking by online courses, due to how do you 'personally' grade someone's submission, especially overtime? Maybe setting it up in a 'there is no right/wrong answer' way, namely because if a student takes a risk, they shouldn't be penalized for it.<p>Like I said, I'd recommend the book. I teach community college currently, and while most this is anecdotal opinion, I feel like it's helped me maintain engaged students.
tkahn6about 12 years ago
I usually sign up for a bunch of classes just so I can access the material at a later point when I have time or when I'm interested. I'm pretty sure that in most cases Coursera locks you out of old material unless you were enrolled. I don't follow along with the class.<p>Synchronous learning, IMO, is an anachronism. We have video recording technology, practically infinite bandwidth, and near speed of light communication. The idea of synchronous learning, where you have five thousand teachers teaching the same Calculus I class every year, is predicated on a world where these technologies don't exist.<p>As an aside, for this reason I think the collegiate model is going to completely change in the near future as people start realizing how absurd it is. The real value professors provide is having someone to answer questions (and in a lot of cases, professors don't even take questions during lecture). And even that functionality can and is (in my experience) mostly replaced by other students in the class or students who have already taken the class.<p>Put the material up, and let communities form around it. I don't need the professor to answer my questions, all I need is the relevant forum (in the original sense) to ask.
评论 #5469738 未加载
评论 #5469555 未加载
评论 #5471995 未加载
评论 #5469678 未加载
评论 #5470014 未加载
评论 #5470581 未加载
评论 #5470166 未加载
michaelochurchabout 12 years ago
The issue with Coursera is the synchronous model. The material's good and the selection's excellent, and they deserve a lot of props for solving such a critical problem. College life is sanitized. Real World interjections are rare enough, at that age, to be handled case-by-case when it comes to extensions and such. Adult life is messy and complicated-- sick parents become more common, job demands fluctuate-- and synchronous education is just brittle. I didn't find Coursera courses to be "too fast". The paces were fine, so long as I didn't have fires in the rest of life.<p>It's not a "dirty secret", though. One should know and expect that. If anything, low retention is a good thing insofar as it means that the courses are demanding and people who aren't dedicated trickle out.<p>I feel like we have three problems to solve in online education. I'm sure there are plenty more, but 4 stand out right now:<p>(1) Hidden node discovery. You're 23. You just learned Python. You want to be a Data Scientist in 3 years. How do you get there? "Data Scientist job" is one node, and there are a bunch of prerequisites to those (hidden nodes) and prerequisites to those. How do you navigate this network? The 23-year-old programmer doesn't know where those hidden nodes are. In other words, the Google for Learning and Development.<p>(2) Forward learning. Recommendations. Things that would be interesting to a person that she doesn't know she wants to know, because she doesn't know that it exists. Since there's a lot of investment here (you're not just buying a book and possibly reading it, but anticipating putting 50+ cognitively intense hours into a course) it would be nice if the service gave indications as to <i>why</i> it was making those rec's.<p>(3) Interactivity and (buzzword warning) "gameification". When you haul out an 800-word machine learning textbook, you often have to go for a long time (hours) without the "kick". There's a flat array of 50 exercises of which 25 are easy, and 25 are really hard and will take a long time (they might be worth doing, but they aren't quick) and it's hard to pick which ones to focus on. The programming exercises in texts often don't get your creative juices flowing. Not a lot of people can get through long spells without feedback, and the skill of "making your own feedback" seems to be losing ground in our distracted culture.<p>(4) "Social". Online study groups. This is Big and I don't know how it's going to evolve. How do we keep quality control in place and make sure that our automated expert discovery mechanisms work?
评论 #5470118 未加载
评论 #5470119 未加载
评论 #5470718 未加载
评论 #5470695 未加载
评论 #5470749 未加载
评论 #5472811 未加载
评论 #5470198 未加载
评论 #5470184 未加载
评论 #5470176 未加载
评论 #5471937 未加载