jpa语句报 org.springframework.dao.InvalidDataAccessApiUsageException: Parameter with that position [2] nested exception is java.lang.IllegalArgumentException: Parameter with that position
时间： 16:24:26
&&&& 阅读：180
&&&& 评论：
&&&& 收藏：0
标签：@Query("SELECT area from Area AS area WHERE area.state=0 AND area.name like %?1% ")Area findByIsCityAndNameLike(int iscity,String name);第一反应: 将 AND area.name like %?1% 后的 1要改成2
但是依然报错仔细反应了 @Query("SELECT area from Area AS area WHERE area.state=0 AND area.isCity = ?1 AND area.name like %?2% ") 就正确了标签：
&&国之画&&&& &&&&chrome插件
版权所有 京ICP备号-2
迷上了代码！小站会根据您的关注，为您发现更多，
看到喜欢的小站就马上关注吧！
下一站，你会遇见谁的梦想？
发布一些iOS开发相关的内容,Xcode开发的一些诀窍，Cocos2d-iPhone的游戏开发的介绍.
iOS判断一个字符串中是否都是数字
第一种方式是使用NSScanner：
1. 整形判断
- (BOOL)isPureInt:(NSString&*)string{
NSScanner* scan = [NSScanner&scannerWithString:string];&
return&[scan&scanInt:&val] && [scan&isAtEnd];
2.浮点形判断：
- (BOOL)isPureFloat:(NSString&*)string{
NSScanner* scan = [NSScanner&scannerWithString:string];&
return&[scan&scanFloat:&val] && [scan&isAtEnd];
第二种方式是使用循环判断
- (BOOL)isPureNumandCharacters:(NSString *)text
& & for(int i = 0; i & [text length]; ++i)&{
& && &&&int a = [text characterAtIndex:i];
& && &&&if ([self isNum:a]){
& && && && &
& && &&&} else {
& && && && &return NO;
& & return YES;
或者 C语言中常用的方式.
- (BOOL)isAllNum:(NSString *)string{& && & for (int i=0; i&string. i++) {& && &&&c=[string characterAtIndex:i];& && &&&if (!isdigit(c)) {& && && && &return NO;& && &&&}& & }& & return YES;}
第三种方式则是使用NSString的trimming方法
- (BOOL)isPureNumandCharacters:(NSString *)string
string = [string stringByTrimmingCharactersInS[NSCharacterSet decimalDigitCharacterSet]];
if(string.length & 0)
& & &return NO;
return YES;
以上三种能够帮助实现判断是否为数字的函数，iOS中没有直接判断是否是数字的方法，所以只能够自己添加方法去实现了.
iOS6屏幕旋转详解（自动旋转、手动旋转、兼容IOS6之前系统）
概述：在iOS6之前的版本中，通常使用&shouldAutorotateToInterfaceOrientation&来单独控制某个UIViewController的方向，需要哪个viewController支持旋转，只需要重写shouldAutorotateToInterfaceOrientation方法。但是iOS 6和iOS 7里屏幕旋转改变了很多，之前的&shouldAutorotateToInterfaceOrientation 被列为&DEPRECATED 方法，查看UIViewController.h文件也可以看到：&
//&Applications&should&use&supportedInterfaceOrientations&and/or&shouldAutorotate.. &-&(BOOL)shouldAutorotateToInterfaceOrientation:(UIInterfaceOrientation)toInterfaceOrientation&NS_DEPRECATED_IOS(2_0,&6_0); &&系统给了如下2个方法来代替：&
-&(BOOL)shouldA &
-&(NSUInteger)supportedInterfaceO &&除了重写这个2个方法，IOS6里面要旋转还有一些需要注意的地方，下面会细述。另外还有一个硬性条件，需要在Info.plist文件里面添加程序支持的所有方向，可以通过以下2种方式添加&1. 在Info.plist的Supported interface orientations的数组里面增加想要的方向，另外，放在第一个的，会在模拟器上默认进入的样式.2. 或者也可以工程的Target的General里面直接选择想要的方向.&另外要兼容IOS6之前的系统，要保留原来的&shouldAutorotateToInterfaceOrientation 方法，还有 willRotateToInterfaceOrientation 等方法。&
IOS6&7自动旋转设置：IOS6以上的系统里面，控制某个viewController旋转并不是像IOS5以下那样在这个viewController里面重写上面那2个方法，而是需要在这个viewController的rootViewController(根视图控制器)里面重写，怎么解释呢？就是最根的那个viewController，直接跟self.window接触的那个controller，比如以下代码：&&
UIViewController&*viewController&=&[[UIViewController&alloc]&init]; UINavigationController&*naviController =&[[UINavigationController&alloc]&initWithRootViewController:&viewController]; &
if&([window&respondsToSelector:@selector(setRootViewController:)])&{
& & & &self.window.rootViewController&=&naviController; &
& & & &[self.window&addSubview:&naviController.view]; &
如果需要设置viewController的旋转，那么不能在UIViewController里面重写shouldAutorotate和supportedInterfaceOrientations方法，而是需要在naviController里面设置，又因为UINavigationController是系统控件，所以这里需要新建一个UINavigationController的子navigationController的子类，然后在里面实现shouldAutorotate和supportedInterfaceOrientations方法，比如：&
-(NSUInteger)supportedInterfaceOrientations{
& & & & &return&UIInterfaceOrientationMaskAllButUpsideD
&-&(BOOL)shouldAutorotate{ &
& & & &return&YES;
} &&例如：如果上面的例子是self.window.rootViewController =&viewController，而不是naviController，那么上面的那2个控制旋转的方法就应该写在UIViewController里面！
例如：如果viewController又pushViewController到viewController2，需要设置viewController2的旋转，怎么办呢？ 还是在naviController里面控制，因为viewController1和viewController2的rootViewController都是naviController，一般的写法都是&
UIViewController&*&viewController2&=&[[UIVewController&alloc]&init]; &[self.navigationController.navigationController&pushViewController:&viewController2&animated:YES]; &&
所以要控制一个UINavigationController push到的所有viewController的旋转，那么就得在naviController里面区分是哪个viewController，以便对他们一一控制！同样如果rootViewController是UITabbarController，那么需要在子类化的UITabbarController里面重写那2个方法，然后分别控制！&
但是有时候我初始化UINavigationController的时候，并不知道所有我所有需要push到的viewController，那么这里有一个通用的方法，就是让viewController自己来控制自己，首先在navCtrl里面的实现方法改为以下方式：&
-&(BOOL)shouldAutorotate
& & & & &return [self.topViewController shouldAutorotate];
(NSUInteger)supportedInterfaceOrientations
& & & & &return [self.topViewController supportedInterfaceOrientations];
} &&全部调用self.topViewController，就是返回当前呈现出来的viewController里面的设置，然后在viewController、viewController2等等这些viewController里面重写shouldAutorotate和supportedInterfaceOrientations，以方便设置每个viewController的旋转
例如：如果viewController&是&presentModalViewController 到&viewController3，那么viewController3的旋转设置就不在naviController里面了！如果presentModalViewController的viewController是naviController、tabbarController包装过的viewController3，那么就应在新包装的navController、tabbarController里面设置，如果是直接presentModalViewController到viewController3，那么就在viewController3里面设置
IOS5以下的自动旋转设置
这个简单很多，没有上面的硬性条件，只需要在需要旋转的viewController里面重写&shouldAutorotateToInterfaceOrientation 方法就行&
手动旋转手动旋转也有2种方式，一种是直接设置&UIDevice 的 orientation，（私有方法，而且测试的时候，第一次有效，第二次设置的时候就会存在问题.）但是这种方式不推荐，上传appStore有被拒的风险：&
if&([[UIDevice&currentDevice]&respondsToSelector:@selector(setOrientation:)]) {
& & & & &[[UIDevice&currentDevice]&performSelector:@selector(setOrientation:)&withObject:(id)UIInterfaceOrientationPortrait];
}&第二种是假旋转，并没有改变&UIDevice 的 orientation，而是改变某个view的&transform，利用&CGAffineTransformMakeRotation&来达到目的，比如：
self.view.transform&=&CGAffineTransformMakeRotation(M_PI/2) &&
下面讲解采用第二种方式的各版本手动旋转：思想是首先设置&statusBarOrientation，然后再改变某个view的方向跟&statusBarOrientation 一致！&
IOS6&7的手动旋转：1. 那既然是旋转，最少也得有2个方向，那么还是少不了上面说的那个硬性条件，先在plist里面设置好所有可能需要旋转的方向。既然是手动旋转，那么就要关闭自动旋转：
-&(BOOL)shouldAutorotate{ &
& & & & return&NO;
}&2.手动触发某个按钮，调用方法，这个方法的实现如下：&
[[UIApplication&sharedApplication]&setStatusBarOrientation:UIInterfaceOrientationLandscapeRight];
self.view.transform&=&CGAffineTransformMakeRotation(M_PI/2); &
self.view.bounds&=&CGRectMake(0,&0,&kScreenHeight,&320); &
注意：1. 只需要改变self.view.transform，那么self.view的所有subview都会跟着自动变；其次因为方向变了，所以self.view的大小需要重新设置，不要使用self.view.frame，而是用bounds。2. 如果shouldAutorotate 返回YES的话，下面设置setStatusBarOrientation 是不管用的！setStatusBarOrientation只有在shouldAutorotate 返回NO的情况下才管用！&
IOS5以下的手动旋转：1.在需要手动旋转的viewController里的 shouldAutorotateToInterfaceOrientation 方法设置 interfaceOrientation == [UIApplicationsharedApplication].statusBarOrientation&
-&(BOOL)shouldAutorotateToInterfaceOrientation:(UIInterfaceOrientation)interfaceOrientation{
& & & & return&(interfaceOrientation&==&[UIApplication&sharedApplication].statusBarOrientation); &} &
2.手动触发某个按钮，调用方法，这个方法的实现如下：&
[[UIApplication&sharedApplication]&setStatusBarOrientation:UIInterfaceOrientationLandscapeRight];
self.view.transform&=&CGAffineTransformMakeRotation(M_PI/2);
self.view.bounds&=&CGRectMake(0,&0,&kScreenHeight,&320); &
注意：只需要改变self.view.transform，那么self.view的所有subview都会跟着自动变；其次因为方向变了，所以self.view的大小需要重新设置，不要使用self.view.frame，而是用bounds。&&
经验分享：1.IOS6&7里面，如果一个项目里面需要各种旋转支持，有自动，有手动，那么我们可以新建2个navController或者tabbarController的子类，一个是不旋转，一个旋转，那么所有需要旋转的UINavigationController都可以用这个子类来代替！包括我们可以定制短信呀、邮件呀的旋转！2.supportedInterfaceOrientations 方法一般是写UIInterfaceOrientationMask方向，但是如果程序要兼容4.3以下的SDK(4.3以下的SDK必须是4.5以下的Xcode，不支持IOS6)，那么在用4.5以下的Xcode编译的时候通不过！所以可以用statusBarOrientation代替或者直接写死数字！&
-(NSUInteger)supportedInterfaceOrientations{
& & & & &return&[UIApplication&sharedApplication].statusBarO &
} &&3.一般都不建议在程序里面直接调用 UIDeviceOrientation 的方向，而是用 UIInterfaceOrientation，他们之间是不同的&
UIInterfaceOrientationLandscapeLeft&=&UIDeviceOrientationLandscapeRight, UIInterfaceOrientationLandscapeRight&=&UIDeviceOrientationLandscapeLeft &&看到吗，左右是反的！Xcode图形化设置方向也是以 UIInterfaceOrientation 为准，就是home按键所在的方向
iOS NSMutableDictionary使用枚举器的时候删除数据出现 Exception
&& & & & 今天使用NSMutableDictionary的时候，使用forin枚举的方式枚举一个字典里面所有的元素，在枚举的时候，如果符合某个条件，就删除这一条数据。& & & & 错误如下：*** Terminating app due to uncaught exception 'NSGenericException', reason: '*** Collection &__NSDictionaryM: 0x& was mutated while being enumerated.'
& & &&代码如下：&
NSMutableDictionary *dic= [NSMutableDictionary&dictionaryWithObjectsAndKeys:
&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&& @"Value1",@"Key1",
&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&& @"Value2",@"Key2",
&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&& @"Value3",@"Key3",
&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&& @"Value4",@"Key4",
&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&& nil];
for (NSString *key in dic)
& &&if([[dic objectForKey: key] isEqualTo:&@"Value2"])
& & & & &[dic removeObjectForKey: key];
}修改如下：
NSArray *keyArr = [dic allKeys];&& & & & & &&for (NSString *key in keyArr)
NSMutableDictionary&*dic= [NSMutableDictionary&dictionaryWithObjectsAndKeys:
&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&@"Value1",@"Key1",
&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&@"Value2",@"Key2",
&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&@"Value3",@"Key3",
&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&@"Value4",@"Key4",
&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&nil];
NSArray&*keyArr = [dic&allKeys];//只要增加这一个就可以解决问题了，相对比较方便
for&(NSString&*key&in&keyArr)//修改dic为keyArr，这样枚举的时候使用的是keyArr，修改的是另一个
& &&if([[dic&objectForKey: key]&isEqualTo:&@"Value2"])
& & & & &[dic&removeObjectForKey: key];
The Tumblr Architecture Yahoo Bought For A Cool Billion Dollars
It's being reported&Yahoo bought Tumblr for $1.1 billion. You may recall&Instagram was profiled on HighScalability&and they were also bought by Facebook for a ton of money. A coincidence? You be the judge.
Just what is Yahoo buying? The business acumen of the deal is not something I can judge, but if you are doing due diligence on the technology then Tumblr would probably get a big thumbs up. To see why, please keep on reading...
With over 15 billion page views a month Tumblr has become an insanely popular blogging platform. Users may like Tumblr for its simplicity, its beauty, its strong focus on user experience, or its friendly and engaged community, but like it they do.&
Growing at over 30% a month has not been without challenges. Some reliability problems among them. It helps to realize that Tumblr operates at surprisingly huge scales: 500 million page views a day, a peak rate of ~40k requests per second, ~3TB of new data to store a day, all running on 1000+ servers.
One of the common patterns across successful startups is the perilous chasm crossing from startup to wildly successful startup. Finding people, evolving infrastructures, servicing old infrastructures, while handling huge month over month increases in traffic, all with only four engineers, means you have to make difficult choices about what to work on. This was Tumblr&s situation. Now with twenty engineers there&s enough energy to work on issues and develop some very interesting solutions.
Tumblr started as a fairly typical large LAMP application. The direction they are moving in now is towards a distributed services model built around Scala, HBase, Redis, Kafka, Finagle, &and an intriguing cell based architecture for powering their Dashboard. Effort is now going into fixing short term problems in their PHP application, pulling things out, and doing it right using services.&
The theme at Tumblr is transition at massive scale. Transition from a LAMP stack to a somewhat bleeding edge stack. Transition from a small startup team to a fully armed and ready development team churning out new features and infrastructure. To help us understand how Tumblr is living this theme is startup veteran&Blake Matheny, Distributed Systems Engineer at Tumblr. Here&s what Blake has to say about the House of Tumblr:
500 million page views a day
15B+ page views month
~20 engineers
Peak rate of ~40k requests per second
1+ TB/day into Hadoop cluster
Many TB/day into MySQL/HBase/Redis/Memcache
Growing at 30% a month
~1000 hardware nodes in production
Billions of page visits per month per engineer
Posts are about 50GB a day. Follower list updates are about 2.7TB a day.
Dashboard runs at a million writes a second, 50K reads a second, and it is growing.
OS X for development, Linux (CentOS, Scientific) in production
PHP, Scala, Ruby
Redis, HBase, MySQL
Varnish, HA-Proxy, nginx,
Memcache, Gearman, Kafka,&Kestrel, Finagle
Thrift, HTTP
Func&- a secure, scriptable remote control framework and API
Git, Capistrano, Puppet, Jenkins
500 web servers
200 database servers (many of these are part of a spare pool we pulled from for failures)
30 memcache servers
22 redis servers
15 varnish servers
25 haproxy nodes
14 job queue servers (kestrel + gearman)
Architecture
Tumblr has a different usage pattern than other social networks.
With 50+ million posts a day, an average post goes to many hundreds of people. It&s not just one or two users that have millions of followers. The graph for Tumblr users has hundreds of followers. This is different than any other social network and is what makes Tumblr so challenging to scale.
#2 social network in terms of time spent by users. The content is engaging. It&s images and videos. The posts aren&t byte sized. They aren&t all long form, but they have the ability. People write in-depth content that&s worth reading so people stay for hours.
Users form a connection with other users so they will go hundreds of pages back into the dashboard to read content. Other social networks are just a stream that you sample.
Implication is that given the number of users, the average reach of the users, and the high posting activity of the users, there is a huge amount of updates to handle.
Tumblr runs in one colocation site. Designs are keeping geographical distribution in mind for the future.
Two components to Tumblr as a platform: public&Tumblelogs&and&Dashboard
Public Tumblelog is what the public deals with in terms of a blog. Easy to cache as its not that dynamic.
Dashboard is similar to the Twitter timeline. Users follow real-time updates from all the users they follow.
Very different scaling characteristics than the blogs. Caching isn&t as useful because every request is different, especially with active followers.
Needs to be real-time and consistent. Should not show stale data. And it&s a lot of data to deal with. Posts are only about 50GB a day. Follower list updates are 2.7TB a day. Media is all stored on S3.
Most users leverage Tumblr as tool for consuming of content. Of the 500+ million page views a day, 70% of that is for the Dashboard.
Dashboard availability has been quite good. Tumblelog hasn&t been as good because they have a legacy infrastructure that has been hard to migrate away from. With a small team they had to pick and choose what they addressed for scaling issues.
Old Tumblr
When the company started on Rackspace it gave each custom domain blog an A record. When they outgrew Rackspace there were too many users to migrate. This is 2007. They still have custom domains on Rackspace. They route through Rackspace back to their colo space using HAProxy and Varnish. Lots of legacy issues like this.
A traditional LAMP progression.
Historically developed with PHP. Nearly every engineer programs in PHP.
Started with a web server, database server and a PHP application and started growing from there.
To scale they started using memcache, then put in front-end caching, then HAProxy in front of the caches, then MySQL sharding. MySQL sharding has been hugely helpful.
Use a squeeze everything out of a single server approach. In the past year they&ve developed a couple of backend services in C: an&ID generator&and&Staircar, using Redis to power Dashboard notifications
The Dashboard uses a scatter-gather approach. Events are displayed when a user access their Dashboard. Events for the users you follow are pulled and displayed. This will scale for another 6 months. Since the data is time ordered sharding schemes don&t work particularly well.
New Tumblr
Changed to a JVM centric approach for hiring and speed of development reasons.
Goal is to move everything out of the PHP app into services and make the app a thin layer over services that does request authentication, presentation, etc.
Scala and Finagle Selection
Internally they had a lot of people with Ruby and PHP experience, so Scala was appealing.
Finagle was a compelling factor in choosing Scala. It is a library from Twitter. It handles most of the distributed issues like distributed tracing, service discovery, and service registration. You don&t have to implement all this stuff. It just comes for free.
Once on the JVM Finagle provided all the primitives they needed (Thrift, ZooKeeper, etc).
Finagle is being used by Foursquare and Twitter. Scala is also being used by Meetup.
Like the Thrift application interface. It has really good performance.
Liked Netty, but wanted out of Java, so Scala was a good choice.
Picked Finagle because it was cool, knew some of the guys, it worked without a lot of networking code and did all the work needed in a distributed system.
Node.js wasn&t selected because it is easier to scale the team with a JVM base. Node.js isn&t developed enough to have standards and best practices, a large volume of well tested code. With Scala you can use all the Java code. There&s not a lot of knowledge of how to use it in a scalable way and they target 5ms response times, 4 9s HA, 40K requests per second and some at 400K requests per second. There&s a lot in the Java ecosystem they can leverage.
Internal services are being shifted from being C/libevent based to being Scala/Finagle based.
Newer, non-relational data stores like HBase and Redis are being used, but the bulk of their data is currently stored in a heavily partitioned MySQL architecture. Not replacing MySQL with HBase.
HBase backs their URL shortner with billions of URLs and all the historical data and analytics. It has been rock solid. HBase is used in situations with high write requirements, like a million writes a second for the Dashboard replacement. &HBase wasn&t deployed instead of MySQL because they couldn&t bet the business on HBase with the people that they had, so they started using it with smaller less critical path projects to gain experience.
Problem with MySQL and sharding for time series data is one shard is always really hot. Also ran into read replication lag due to insert concurrency on the slaves.
Created a common services framework.
Spent a lot of time upfront solving operations problem of how to manage a distributed system.
Built a kind of Rails scaffolding, but for services. A template is used to bootstrap services internally.
All services look identical from an operations perspective. Checking statistics, monitoring, starting and stopping all work the same way for all services.
Tooling is put around the build process in&SBT&(a Scala build tool) using plugins and helpers to take care of common activities like tagging things in git, publishing to the repository, etc. Most developers don&t have to get in the guts of the build system.
Front-end layer uses HAProxy. Varnish might be hit for public blogs. 40 machines.
500 web servers running Apache and their PHP application.
200 database servers. Many database servers are used for high availability reasons. Commodity hardware is used an the MTBF is surprisingly low. Much more hardware than expected is lost so &there are many spares in case of failure.
6 backend services to support the PHP application. A team is dedicated to develop the backend services. A new service is rolled out every 2-3 weeks. Includes dashboard notifications, dashboard secondary index, URL shortener, and a memcache proxy to handle transparent sharding.
Put a lot of time and effort and tooling into&MySQL sharding. MongoDB is not used even though it is popular in NY (their location). MySQL can scale just fine..
Gearman, a job queue system, is used for long running fire and forget type work.
Availability is measured in terms of reach. Can a user reach custom domains or the dashboard? Also in terms of error rate.
Historically the highest priority item is fixed. Now failure modes are analyzed and addressed systematically. Intention is to measure success from a user perspective and an application perspective. If part of a request can&t be fulfilled that is account for
Initially an Actor model was used with Finagle, but that was dropped. &For fire and forget work a job queue is used. In addition, Twitter&s&utility library&contains a&Futuresimplementation and services are implemented in terms of futures. In the situations when a thread pool is needed futures are passed into a future pool. Everything is submitted to the future pool for asynchronous execution.
Scala encourages no shared state. Finagle is assumed correct because it&s tested by Twitter in production. Mutable state is avoided using constructs in Scala or Finagle. No long running state machines are used. State is pulled from the database, used, and writte n back to the database. Advantage is developers don&t need to worry about threads or locks.
22 Redis servers. Each server has 8 - 32 instances so 100s of Redis instances are used in production.
Used for backend storage for dashboard notifications.
A notification is something &like a user liked your post. Notifications show up in a user&s dashboard to indicate actions other users have taken on their content.
High write ratio made MySQL a poor fit. &
Notifications are ephemeral so it wouldn&t be horrible if they were dropped, so Redis was an acceptable choice for this function.
Gave them a chance to learn about Redis and get familiar with how it works.
Redis has been completely problem free and the community is great.
A Scala futures based interface for Redis was created. This functionality is now moving into their Cell Architecture.
URL shortener uses Redis as the first level cache and HBase as permanent storage.
Dashboard&s secondary index is built around Redis.
Redis is used as Gearman&s persistence layer using a memcache proxy built using Finagle.
Slowly moving from memcache to Redis. Would like to eventually settle on just one caching service. Performance is on par with memcache.
Internal Firehose
Internally applications need access to the activity stream. An activity steam is information about users creating/deleting posts, liking/unliking posts, etc. &A challenge is to distribute so much data in real-time. Wanted something that would scale internally and that an application ecosystem could reliably grow around. A central point of distribution was needed.
Previously this information was distributed using Scribe/Hadoop. Services would log into Scribe and begin tailing and then pipe that data into an app. This model stopped scaling almost immediately, especially at peak where people are creating 1000s of posts a second. Didn&t want people tailing files and piping to grep.
An internal firehose was created as a message bus. Services and applications talk to the firehose via Thrift.
LinkedIn&s Kafka is used to store messages. Internally consumers use an HTTP stream to read from the firehose. MySQL wasn&t used because the sharding implementation is changing frequently so hitting it with a huge data stream is not a good idea.
The firehose model is very flexible, not like Twitter&s firehose in which data is assumed to be lost.
The firehose stream can be rewound in time. It retains a week of data. On connection it&s possible to specify the point in time to start reading.
Multiple clients can connect and each client won&t see duplicate data. Each client has a client ID. Kafka supports a consumer group idea. Each consumer in a consumer group gets its own messages and won&t see duplicates. Multiple clients can be created using the same consumer ID and clients won&t see duplicate data. This allows data to be processed independently and in parallel. Kafka uses ZooKeeper to periodically checkpoint how far a consumer has read.
Cell Design For Dashboard Inbox
The current scatter-gather model for providing Dashboard functionality has very limited runway. It won&t last much longer.
The solution is to move to an inbox model implemented using a Cell Based Architecture that is similar to&Facebook Messages.
An inbox is the opposite of scatter-gather. A user&s dashboard, which is made up posts from followed users and actions taken by other users, &is logically stored together in time order.
Solves the scatter gather problem because it&s an inbox. You just ask what is in the inbox so it&s less expensive then going to each user a user follows. This will scale for a very long time.
Rewriting the Dashboard is difficult. The data has a distributed nature, but it has a transactional quality, it&s not OK for users to get partial updates.
The amount of data is incredible. Messages must be delivered to hundreds of different users on average which is a very different problem than Facebook faces. Large date + high distribution rate + multiple datacenters.
Spec&ed at a million writes a second and 50K reads a second. The data set size is 2.7TB of data growth with no replication or compression turned on. The million writes a second is from the 24 byte row key that indicates what content is in the inbox.
Doing this on an already popular application that has to be kept running.
A cell is a self-contained installation that has all the data for a range of users. All the data necessary to render a user&s Dashboard is in the cell.
Users are mapped into cells. Many cells exist per data center.
Each cell has an HBase cluster, service cluster, and Redis caching cluster.
Users are homed to a cell and all cells consume all posts via firehose updates.
Each cell is Finagle based and populates HBase via the firehose and service requests over Thrift.
A user comes into the Dashboard, users home to a particular cell, a service node reads their dashboard via HBase, and passes the data back.
Background tasks consume from the firehose to populate tables and process requests.
A Redis caching layer is used for posts inside a cell.
Request flow: a user publishes a post, the post is written to the firehose, all of the cells consume the posts and write that post content to post database, the cells lookup to see if any of the followers of the post creator are in the cell, if so the follower inboxes are updated with the post ID.
Advantages of cell design:
Massive scale requires parallelization and parallelization requires components be isolated from each other so there is no interaction. Cells provide a unit of parallelization that can be adjusted to any size as the user base grows.
Cells isolate failures. One cell failure does not impact other cells.
Cells enable nice things like the ability to test upgrades, implement rolling upgrades, and test different versions of software.
The key idea that is easy to miss is:&&all posts are replicated to all cells.
Each cell stores a single copy of all posts. Each cell can completely satisfy a Dashboard rendering request. Applications don&t ask for all the post IDs and then ask for the posts for those IDs. It can return the dashboard content for the user. Every cell has all the data needed to fulfill a Dashboard request without doing any cross cell communication.
Two HBase tables are used: one that stores a copy of each post. That data is small compared to the other table which stores every post ID for every user within that cell. The second table tells what the user&s dashboard looks like which means they don&t have to go fetch all the users a user is following. It also means across clients they&ll know if you read a post and viewing a post on a different device won&t mean you read the same content twice. With the inbox model state can be kept on what you&ve read.
Posts are not put directly in the inbox because the size is too great. So the ID is put in the inbox and the post content is put in the cell just once. This model greatly reduces the storage needed while making it simple to return a time ordered view of an users inbox. The downside is each cell contains a complete copy of call posts. Surprisingly posts are smaller than the inbox mappings. Post growth per day is 50GB per cell, inbox grows at 2.7TB a day. Users consume more than they produce.
A user&s dashboard doesn&t contain the text of a post, just post IDs, and the majority of the growth is in the IDs.
As followers change the design is safe because all posts are already in the cell. If only follower posts were stored in a cell then cell would be out of date as the followers changed and some sort of back fill process would be needed.
An alternative design is to use a separate post cluster to store post text. The downside of this design is that if the cluster goes down it impacts the entire site. &Using the cell design and post replication to all cells creates a very robust architecture.
A user having millions of followers who are really active is handled by selectively materializing user feeds by their access model (see&Feeding Frenzy).
Different users have different access models and distribution models that are appropriate. Two different distribution modes: one for popular users and one for everyone else.
Data is handled differently depending on the user type. Posts from active users wouldn&t actually be published, posts would selectively materialized.
Users who follow millions of users are treated similarly to users who have millions of followers.
Cell size is hard to determine. The size of cell is the impact site of a failure. The number of users homed to a cell is the impact. There&s a tradeoff to make in what they are willing to accept for the user experience and how much it will cost.
Reading from the firehose is the biggest network issue. Within a cell the network traffic is manageable.
As more cells are added cells can be placed into a cell group that reads from the firehose and then replicates to all cells within the group. A hierarchical replication scheme. This will also aid in moving to multiple datacenters.
On Being A Startup In New York
NY is a different environment. Lots of finance and advertising. Hiring is challenging because there&s not as much startup experience.
In the last few years NY has focused on helping startups. NYU and Columbia have programs for getting students interesting internships at startups instead of just going to Wall Street. Mayor Bloomberg is establishing a local campus focused on technology.
Team Structure
Teams: infrastructure, platform, SRE, product, web ops, services.
Infrastructure: Layer 5 and below. IP address and below, DNS, hardware provisioning.
Platform: core app development, SQL sharding, services, web operations.
SRE: sits between service team and web ops team. Focused on more immediate needs in terms of reliability and scalability.
Service team: focuses on things that are slightly more strategic, that are a month or two months out.
Web ops: responsible for problem detection and response, and tuning.
Software Deployment
Started with a set of rsync scripts that distributed the PHP application everywhere. Once the number of machines reached 200 the system started having problems, deploys took a long time to finish and machines would be in various states of the deploy process.
The next phase built the deploy process (development, staging, production) into their service stack using Capistrano. Worked for services on dozens of machines, but by connecting via SSH it started failing again when deploying to hundreds of machines.
Now a piece of coordination software runs on all machines. Based around Func from RedHat, a lightweight API for issuing commands to hosts. Scaling is built into Func.
Build deployment is over Func by saying do X on a set of hosts, which avoids SSH. Say you want to deploy software on group A. The master reaches out to a set of nodes and runs the deploy command.
The deploy command is implemented via Capistrano. It can do a git checkout or pull from the repository. Easy to scale because they are talking HTTP. They like Capistrano because it supports simple directory based versioning that works well with their PHP app. Moving towards versioned updates, where each directory contains a&SHA&so it&s easy to check if a version is correct.
The Func API is used to report back status, to say these machines have these software versions.
Safe to restart any of their services because they&ll drain off connections and then restart.
All features run in dark mode before activation.
Development
Started with the philosophy that anyone could use any tool that they wanted, but as the team grew that didn&t work. Onboarding new employees was very difficult, so they&ve standardized on a stack so they can get good with those, grow the team quickly, address production issues more quickly, and build up operations around them.
Process is roughly Scrum like. Lightweight. &
Every developer has a preconfigured development machine. It gets updates via Puppet.
Dev machines can roll changes, test, then roll out to staging, and then roll out to production.
Developers use vim and Textmate.
Testing is via code reviews for the PHP application.
On the service side they&ve implemented a testing infrastructure with commit hooks, Jenkins, and continuous integration and build notifications.&
Hiring Process
Interviews usually avoid math, puzzles, and brain teasers. Try to ask questions focused on work the candidate will actually do. Are they smart? Will they get stuff done? But measuring &gets things done& is difficult to assess. Goal is to find great people rather than keep people out.
Focused on coding. They&ll ask for sample code. During phone interviews they will use Collabedit to write shared code.
Interviews are not confrontational, they just want to find the best people. Candidates get to use all their tools, like Google, during the interview. The idea is developers are at their best when they have tools so that&s how they run the interviews.
Challenge is finding people that have the scaling experience they require given Tumblr&s traffic levels. Few companies in the world are working on the problems they are.
Example, for a new ID generator they needed A JVM process to generate service responses in less the 1ms at a rate at 10K requests per second with a 500 MB RAM limit with High Availability. They found the serial collector gave the lowest latency for this particular work load. Spent a lot of time on JVM tuning.
On the Tumblr Engineering Blog they&ve posted memorials giving their respects for the passing of&Dennis Ritchie&&&John McCarthy. It&s a geeky culture.
Lessons Learned
Automation everywhere.
MySQL (plus sharding) scales, apps don't.
Redis is amazing.
Scala apps perform fantastically.
Scrap projects when you aren&t sure if they will work.
Don&t hire people based on their survival through a useless technological gauntlet. &Hire them because they fit your team and can do the job.
Select a stack that will help you hire the people you need.
Build around the skills of your team.
Read papers and blog posts. Key design ideas like the cell architecture and selective materialization were taken from elsewhere.
Ask your peers. They talked to engineers from Facebook, Twitter, LinkedIn about their experiences and learned from them. You may not have access to this level, but reach out to somebody somewhere.
Wade, don&t jump into technologies. They took pains to learn HBase and Redis before putting them into production by using them in pilot projects or in roles where the damage would be limited.
I&d like to thank Blake very much for the interview. He was very generous with his time and patient with his explanations. Please&contact me&if you would like to talk about having your architecture profiled.
Related Articles
Tumblr Engineering Blog&- contains a lot of good articles
Building Network Services with Finagle and Ostrich&by Nathan Hamblen - Finagle is awesome
Ostrich&- A stats collector & reporter for Scala servers
ID Generation at Scale&by Blake Matheny
Finagle&- a network stack for the JVM that you can use to build asynchronous Remote Procedure Call (RPC) clients and servers in Java, Scala, or any JVM-hosted language. Finagle provides a rich set of protocol-independent tools.
Finagle Redis client from Tumblr
Tumblr. &Massively Sharded MySQL&by Evan Elias - one of the better presentations on MySQL sharding available
Staircar: Redis-powered notifications&by Blake Matheny
Flickr Architecture&- talks about Flickr's cell architecture
将你的代码迁移到Objective-C ARC（译）
& & & &最近，苹果公司提出了几个给开发者的东西包括 Xcode4、ARC、LLVM Compiler 3.0和iOS5.0。通过在Statck overflow（一个与语言无关，让程序员可以提出问题或者回答问题的网站，网址：/）的一些问题，我可以知晓，大多数的关于ARC的困惑的增加源于一个事实&&开发者不清楚&ABC&（表示这一个单词不理解，Automatic Block Counting？）是一个LLVM 3.0的或者iOS 5的或者ARC的特性/限制。& & & Retain cycles(引用计数环)，auto-release pools（自动回收池），@autorelease blocks（自动释放块）！天哪！这么多新的东西，我该怎么做啊？没错，坦诚地说，ARC，或者说Objective-C Automatic Reference Counting（Objective-C自动引用计数）已经和iPad一样神奇了。& & & 在这篇帖子里，我已经尽最大努力揭开ARC的神秘面纱。但是在开始之前，我必须提醒你，这是一篇相当长的帖子。如果你已经很厌烦了，保存（Instapaper，在线书签，是一款网页脱机阅读软件，它能保存你未能阅读的网页，以供你今后离线浏览，包含电脑，ios多种应用。网址：/）这篇文章留待以后阅读。但是，很幸运的，我相信在阅读完这篇文章之后，你会对ARC如何工作有一个更深的理解，并且有能力解决在转换你的工程的时候ARC抛出的无数错误。& & & 已经说了这么多了，让我们开始吧~目录：&一、什么是ARC& & & ARC是新LLVM 3.0编译器的一个特性，它能够帮助你放心的写你的代码而不用过多的担心内存管理。内存管理可以被广义的分为两种机制：垃圾回收机制和引用计数机制。在深入讨论之前，让我们简要的讨论一下这两类机制并理解为什么对ARC的需要这么迫切。& & & 1、当前机制的问题& & & & & & & 当前在Objective-C中使用的内存管理模式是在iOS上手动引用计数，在Mac上使用垃圾回收机制。这两种内存管理机制都有一定的问题，这个问题同时也是ARC被开发出来的原因。& & & & & & & ①、垃圾回收& & & & & & & & & & 垃圾回收是一个更高层次的语言特性大概是从Java中引入的（或者更专业的，Java Virtual Machine，Java虚拟机）&&
Migrating your code to Objective-C ARC
Recently, Apple introduced several new developer stuff including Xcode 4, ARC, LLVM Compiler 3.0 and iOS 5. From some of the questions on Stack overflow, I could understand that, most of the ARC related confusions arise due to the fact that, developers don&t know if &ABC& is a feature/restriction of LLVM 3.0 or iOS 5 or ARC.Retain cycles, auto-release pools, @autorelease blocks, oh man! So many new things? What am I going to do? You are right. ARC, or Objective-C Automatic Reference Counting is almost&. No really.In this post, I&ve made an attempt to demystify the air around this. Before starting, I&ll have to warn you that, this is a fairly long post. If you are too bored, Instapaper this article and read it later. But, hopefully, at the end of this, I believe, you will have a better understanding on how ARC works and be able to work around the innumerable errors it spits out when you convert your project.Having said that, let&s get started.Contents
What is ARC
Problems with the current model.
Garbage collection
Reference Counting
Automatic Reference Counting
Compiler level feature
Also a run time feature
ARC Ownership qualifiers
ARC knows more Objective-C than you
Toll-free bridging
How does ARC work internally?
ARC front end
ARC optimizer
The actual Migration using Xcode 4.2
Common ARC migration errors
Cast of Objective-C pointer to C Pointer type
performSelector may cause a leak because its selector is unknown
Receiver type "*" doesn't declare the method with selector "*"
Common workarounds that you use in ARC on code that otherwise looks normal
Capturing "*" strongly is likely to lead to a retain cycle
Avoiding early releases using __block
That last question
When should you migrate?
Where to go from here?&What is ARCARC is a feature of the new LLVM 3.0 compiler that helps you to write code without worryingmuch&about memory management. Memory management can be broadly classified into two, garbage collected and reference counted models. Before going to the details, let&s briefly discuss these two models and understand why ARC is even needed.Problems with the current model.The current memory model we use in Objective-C is manual reference counting on iOS and Garbage collection on Mac.There are certain problems with both these memory models which probably was the reason why ARC was developed.Garbage collectionGarbage collection is a higher level language feature probably introduced in Java (or technically, Java Virtual Machine) and implemented in a variety of other programming platforms including Microsoft&s Common Language Runtime. While Garbage collection worked well for higher level languages, Objective-C, which is still C under the hood, didn&t really fly high. Pointers (or rather references) in other languages like Java were actually objects that managed retain count and automatically releases itself when the count reaches zero. One of the design goals of C was to be optimized for performance and not &easy of use&. While pointer objects (read&) are great object oriented abstractions, they have an adverse effect on the performance of the code and since Objective-C was intended primarily for native programming where developers are used to use pointers, pointers within a structure, pointer to a pointer (for dereferencing a out parameter), it was just too difficult to introduce something like a smart pointer that would require a lot of mindset change from the developers who prefer a deterministic memory management model (Reference counting) over a non-deterministic memory management model (Garbage collection). Nevertheless, GC (Generational GC) was introduced in Objective-C 2.0 for Mac. While Generational GC doesn&t suffer from &Stop the world& issues like the mark and sweep alogrithm, they don&t collect every released variable and an occasional mark and sweep collection is still needed.Reference CountingThe memory management model used in iOS is called as reference counting model, or more precisely, manual reference counting.In manual reference counting model, you as a developer, have to deallocate every object you allocated. When you don&t do this, you either leak memory or over release it, causing a crash. While that counting sounds easy, Most of the memory leaks happen when you transfer ownership of objects across scope boundaries. That&s a creator method that allocates an object for you and expects the caller to deallocate it. To circumvent this problem, Objective-C introduced a concept called autorelease. auto-released variables are added to the auto-release pool and are released at the end of the runloop. While this sounds too good, auto-release pools do incur an additional overhead.For example, comparing the two code blocks,
NSDictionary *dict = [[NSDictionary alloc] init];
// do something with the dictionary here
// I'm done with the dictionary, I don't need it anymore
[dict release];
// more code}
NSDictionary *dict = [NSDictionary dictionary]; // auto-released object
// do something with the dictionary here
// I'm done with the dictionary, I don't need it anymore&
// more code}
the first block is an example of optimized use of memory where as the second depends on auto-release pools. While this block of code doesn&t really incur significant memory overhead, code like these slowly adds together and makes your reference counted model heavily dependent on auto-release pools. That is, objects that you know could be deallocated, will still linger around in the auto-release pool for a little longer.Automatic Reference CountingSay hello to ARC. ARC is a compiler feature that auto inserts retain and release for you. So in the first code block of the above example, you no longer have to write the release method and ARC auto-inserts for you before compilation.
NSDictionary *dict = [[NSDictionary alloc] init];
// do something with the dictionary here
// I'm done with the dictionary, I don't need it anymore
[dict release]; // ARC inserts this code for you.
// more code}
When you create an autoreleased object, like in the second block of code, ARC compiler is clever enough not to add a release call. Sound great, so how should I go about doing this ARC thing? Just delete all release/retain codes and pray? Unfortunately, it isn&t that easy. ARC is not just some auto insert or macro expander kind of tool.&It forces you to think in terms of object graphs instead of memory allocation, retain or release.&Let&s delve a little deeper into ARC.Compiler level featureARC is a compiler level feature. I repeat. ARC IS A COMPILER LEVEL FEATURE. This means, when you use ARC, you don&t have to worry about upgrading your deployment target and so on. However, only the latest LLVM 3.0 compiler supports ARC. If you are still stuck with GCC, you are out of luck. (Meh!) Some more points that you should know about ARC are,
ARC is backward compatible with libraries and framework compiled with under non-ARC.
ARC can be used within your project on a file-by-file basis. So you can mix and match ARC code with non-ARC code.
You can also integrate ARC compiled libraries into your project that doesn&t use ARC and vice-versa.
You use a&compiler switch&to turn ARC on and off.
(The keyword here is compiler switch)
You can also set the complete target to build with ARC by default (and use non-ARC compiler only when instructed so.) This is shown in the illustration below.
The two main compiler switches that you would often use are when you build your application with a third party library that is not ARC compliant and vice versa, are
-fno-objc-arc
-fobjc-arc
-f is the switch and&no-objc-arc&and&objc-arc&are the options that you are turning on. As evident from the names, the first one turns off ARC and the second turns on.For example, if your application is ARC enabled but a third party library is not, you use the first switch&-fno-objc-arcto exclude the third party library. Conversely, if your application is not yet ARC enabled (gasp!) but the third party library you are integrating is, you use the second switch&-fobjc-arc&You add these flags to the project from the Build phases tab as shown below.Also a run time featureWait! You just told me (and repeated) that ARC is a compiler level feature? Now what? Sorry, I hear you, but, unfortunately, things aren&t that easy and it doesn&t just stop here. ARC also backs up on a runtime feature calledzero-ing weak references. Oh, damn, another keyword! I should have introduced this before. But that&s ok. We will revisit about the run-time dependency of ARC, a little later in this post.ARC Ownership qualifiersAs I showed you earlier, ARC automatically inserts releases and retains in your code in a pre-compilation step. But for ARC to know when to release your objects and when to retain them, you need to somehow tell the life of your variables. You use ownership qualifiers for that. A strong understanding of ownerships is vital to understand and use ARC properly. Once you understand this concept, you will be thinking in terms of object graphs instead of retain/release. Secondly, when you use ARC, all variables local or ivars are initialized to nil automatically for you. This means, there is little chance of having a dangling reference in your application.
__unsafe_unretained
__autoreleasingThe first qualifier, __strong, is the default and you might not even be using this explicitly. It is use to tell the ARC compiler that, the declared variable &owns& the reference. The opposite of this is __weak, which tells the ARC compiler that the declared variable doesn&t own the reference.The __weak is&synonymous&to the &assign& modifier. You normally use assign modifier for IBOutlets and delegates. Under ARC, this is replaced with __weak. However, there is a caveat. __weak requires you to deploy the app on a runtime that supports&zero-ing weak references. This includes, iOS 5 and Lion. Snow Leopard and older operating systems or iOS 4 and older operating systems don&t support zero-ing weak references. This obviously means you cannot use __weak ownership modifiers if you plan to deploy to older operating systems. Fret not. ARC includes another ownership qualifier, __unsafe_unretained that is synonymous to __weak, except that when the pointer is de-referenced, it is not set to nil, but remains dangling. A while ago, I told something about zero-ing weak references? When the runtime supports zero-ing weak references, your __weak variables are automatically set to nil when they are released. This is the only feature that requires a higher deployment target (iOS 5/Lion). Otherwise, you are good to deploy on iOS 4/Snow Leopard.A couple other important things to know about __weak vs __unsafe_unretained is that, the compiler doesn&t allow you to use __weak when your deployment target is set to a operating system that doesn&t support zero-ing weak references. The Convert to Objective-C ARC wizard uses __weak only when your deployment target supports zero-ing weak references. So if your deployment target is iOS 4, the Objective-C convert ion wizard will replace assign modifiers with __unsafe_unretained instead of __weak.The last ownership qualifier, __auto_releasing is used mostly when passing a reference to a function for writing out. You would use this in places where you normally use pointer indirection like returning a NSError object via an out parameter.Properties in your header file can also have the above ownership qualifiers except the __auto_releasing. When applied to properties, ARC automatically generates the correct code in dealloc to release them when the object dies.Lastly, and more importantly, all of ARC managed&objects&are initialized to nil when they are created. So, again, no more dangling pointers because you forgot a initialize statement. However, do note that this initialization doesn&t initialize primitive data types. So a declaration like,
might contain a garbage value for a.Whew! That&s pretty taxing. Take a break. We just started.ARC knows more Objective-C than youARC also taps into a the Objective-C language naming conventions and infers the ownership of the returned object.In Objective-C, a method that stats with any one of the following prefix,
mutableCopy and
neware considered to be transferring ownership of the returned object to the caller.This means, in your application, when you create a method, ARC automatically infers whether to return a autoreleased object or a +1 retained object from your method name. In fact, in most cases, instead of returning auto-release objects, ARC just inserts a manual release in the calling code, automatically for you. However, there is a small caveat. Let&s assume that you have a method that starts with &copy&, as in
-(NSString*) copyRightS
ARC assumes that it would transfer the ownership of the returned string to the caller and inserts a release automatically. Everything works well, if both the called method and the calling method are compiled using ARC.
But&if your &copyRightString& method is in a third party library that isn&t compiled with ARC, you will over-release the returned string. This is because, on the calling code, ARC compiler inserts a release to balance out the retain count bumped up by the &copy& method. Conversely, if the third party library is compiled with ARC and your method isn&t, you will have a memory leak. You can however override this behavior by adding one of the following attribute to your methods.
NS_RETURNS_NON_RETAINED
NS_RETURNS_RETAINEDSo your method will now look like this.
-(NSString*) copyRightString NS_RETURNS_NON_RETAINED;
You can also rename the method name to&copyrightString&(note the case) or&getCopyRightString&instead of adding an attribute. However, I wouldn&t recommend the former method as it breaks the cocoa naming conventions (prefixing a method with &get& or &set& is Java-ish)You will see methods having the NS_RETURNS_* prefixes throughout the header files in Apple&s own UIKit.framework or the foundation classes. Now that you know what happens behind the scenes and how compiler treats these decorations, you can solve crazy memory issues, like a crash when you call a&copyRightString&in your method in a third party library.With that, let&s get ready for climbing the next peak.Toll-free bridgingARC&doesn&t&manage Core Foundation objects. They say, there is no free lunch. ARC, takes it one step further. There is no free-casting between Core Foundation objects and equivalent Objective-C objects (NS* objects). Yes, that&s right. You cannot cast a Core Foundation object to an equivalent Objective-C object (NS* object) without telling ARC how to manage ownerships.Let&s now see how to specify ownership transfers when you cast a Core Foundation object.The following ownership transfer modifiers should be provided when you cast a Objective-C object to a Core Foundation object.
__bridge_retained
__bridge_transferWhen you migrate a project to ARC, you would have seen error messages like the one below.
ARC Error because of a missing bridge attribute in a Toll-free bridging codeYou might also have proceeded by accepting the suggestions provided by the LLVM compiler. But now, let&s dig deeper and understand the &why& behind it.The modifier, __bridge tells the ARC compiler that, it&s a plain simple, bridging cast. That means, you ask the ARC compiler to do nothing extra when the transfer is made. You might think, if that is the case, Apple could have made this the default choice. But it was not made probably because, it&s to preposterous to make such an assumption. Making such a bold assumption means, you would easily leak memory as there isn&t a easier way to tell when you are actually releasing a Core Foundation object unlike a Objective-C object.The second modifier, __bridge_retained is used to tell the ARC compiler that the Objective-C object should be transferred to Core Foundation by bumping the retain count by 1 and it should be treated as if it is a newly created object (as opposed to a auto-released object). You use this modifier if the method was probably named like a creation method (starting with init, copy, mutableCopy etc.,) or if you are going to release the Objective-C object inside of Core Foundation using methods like CFRelease.The last modifier, __bridge_transfer is used to tell the ARC compiler that the Core Foundation object is to be transferred to ARC with a retain count of 1. This is used if you created a Core Foundation object using one of the CF***Create methods and want the ARC compiler to handle the memory management for you. That&s you are transferring a Core Foundation object to ARC with a retain count of 1.As a side note on this, avoid using __bridge_retained and __bridge_transfer to trick the compiler to add retain and releases for you. Use it to improve your code readability and minimizing the number of manual memory management calls. (Move on if you don&t understand this line. You will start understanding this automatically when you start using this in your own code)How does ARC work internally?ARC ain&t magic, if you know how it works. But a little knowledge is a dangerous thing. Knowing how the ARC compiler works will help you more in understanding the error messages and compiler warnings spat out by it.The ARC compiler has two main parts, a front end compiler and an optimizer.ARC front endThe ARC front end compiler checks for every &owned& Objective-C object and inserts release appropriately. By owned object, I mean, an object whose ownership qualifier has been set. For example, if the &owned& object is a local variable, ARC front end compiler inserts a release at the end of the scope. This is because, by default all local variables are &strong& ly owned. If the object is a instance variable, the ARC front end compiler inserts a release statement in the dealloc method, if the ownership type is strong. For unsafe_unretained or weak ownership ARC doesn&t do anything. It also takes care of calling the [super dealloc] for you and intact ARC compiler doesn&t allow you to explicitly call dealloc.The ARC front end compiler also takes care of generating errors when it encounters a variable (local or instance) whose ownership qualifier is not set or when you explicitly calling dealloc.ARC optimizerThe function of the ARC optimizer is to optimize the retain and release statements by removing them if they are inserted multiple times by the ARC front end compiler. It is this optimizer that ensures that performance is not affected by calling retain and release multiple times.The actual Migration using Xcode 4.2Xcode 4.2 has a wizard to automatically migrate your code for use with the ARC compiler. This means, the wizard rewrites some of your code, removes calls to retain/release and removes dealloc methods and calls to [super dealloc] for you.The first step is to open your project, select Edit -& Refactor -& Convert to Objective-C ARC from the menu.
Migrating to Objective-C ARC using Xcode 4.2When you select this option, you will be asked to select a target. If you have only one target, it&s fine. If you have multiple targets in your application, you have to perform the ARC migration on every target. After you select a target, the wizard by default selects all source code files that belong to that project for ARC migration. If you are using third party libraries that are not yet ARC ready, you can uncheck those files in this step. This is illustrated in the screenshot below.
Selecting your files for ARC exclusionIn the above project, since I know that ASIHttpRequest is not yet ARC compatible, I&m selecting them and command-clicking them to show the option to uncheck all of them. When you do this, the wizard automatically adds a -fno-objc-arc compiler flag for all these files.The next step is to start the pre-checking process. The pre-checking process compiles the project and analyzes for potential problems before performing the actual migration. You might almost and always get a error message like this.
The dreaded error message!Of course, 58 errors in this screenshot is actually quite low. You should expect anywhere in the range of 300+ for a mid sized project. But fret not, they aren&t complicated at all to fix.Common ARC migration errorsThe number of errors that might prevent you from converting your project to ARC is usually high if your code is &old& or if it doesn&t adhere to Objective-C design patterns. For example, accessing a iVar. While it&s technically ok, you should almost and always use properties to access them outside of init and dealloc methods. If you have been using properties, ARC migration would be painless. If you were old skool, you have to feel the pain now. In this last section, I&ll show you the most commonly occurring errors when you migrate your project.Cast of Objective-C pointer to C Pointer typeThis error is generated because ARC doesn&t do toll-free bridging for you. As I explained before in the section, Toll-free bridging, requires the developer to explicitly specify ownership transfer qualifiers.Use the various ownership transfer qualifiers I showed you before to fix this problem.performSelector may cause a leak because its selector is unknownWe now know that Objective-C ARC compiler knows more Objective-C than you. This error message is because of that. The ARC compiler tries to identify the method family and determine whether to add a retain or release to the returned value from the caller code. This means, if your method starts with init, alloc, copy, mutableCopy or new, the ARC compiler will add a release to the calling code after the variable scope ends. Since are using a selector to call a method dynamically at runtime, ARC doesn&t really know if the method called returns a +1 retained object or a auto-released object. As such, ARC cannot reliably insert a retain or release to the returned object after its scope ends. This warning is shown to warn you of potential memory leaks.If you are sure that your code works fine without memory leaks, you can ignore this warning. To suppress this warning, you can turn off the compiler flag -Warc-performSelector-leaks warning on a line by line basis like this.
#pragma clang diagnostic push#pragma clang diagnostic ignored "-Warc-performSelector-leaks"
[self performSelector:self.mySel]; #pragma clang diagnostic pop
Unfortunately, you cannot annotate a dynamic selector using __attribute__ ((objc_method_family(*))).Receiver type &*& doesn&t declare the method with selector &*&ARC mandates that every method you call should be declared properly. With GCC, this was a warning. But LLVM makes this step mandatory since ARC needs to accurately identify and that is why you see an error like this.
Error that you see when you don't declare a receiver typeThis error is also because of the fact that ARC needs to identify the method family to determine if it has to add a retain or release to the returned object. For example, in the above code, the method, returnMyGreatObject might return a NS_RETURNS_RETAINED. In this case, the ARC compiler has to insert a release after the returned object goes out of scope. The ARC compiler can know this only when you declare it formally. This is why, under ARC method declarations are mandatory. If you have been declaring methods formally under GCC, even when the compiler didn&t enforce (so that the code was aesthetically beautiful) you wouldn&t see this error at all. As I said before, the number of ARC migration errors is directly proportional to the quality of code you write. Fixing this error is fairly simple and all you have to do is to declare every method formally in the header file or on a private category extension.Common workarounds that you use in ARC on code that otherwise looks normalIn some cases, while using ARC, you might end up writing code that looks as if it&s written to &please& the ARC compiler rather than writing natural code. Unfortunately, nothing can be done to this and we all have to live with this. The next two sections explain when you might need to write unnecessary code like this to please the compiler.Capturing &*& strongly is likely to lead to a retain cycle
ARC and retain cyclesThe last category of warning message is shown when a retain cycle is detected in your code. An example is shown below.This code was probably leaking the request object before ARC and increasing your memory footprint. But, thanks to ARC. You now know that code like these cause retain cycles that cannot be released automatically. Circumventing a retain cycle issue almost and always ends up breaking the cycle with a weak reference.Fixing this error is fairly simple and in this case, you can get a weak reference to the request object and copy it to the block. Within the block, convert it again to a strong reference. This is illustrated below.
Workaround for ARC and retain cycle issueIn the above code block, you can also replace references to __unsafe_unretained with __weak if you are deploying to a runtime that supports zero-ing weak references.Avoiding early releases using __blockSometimes, you need an object to live till as long as the completion handler on it can live. For example, a Block based UIAlertView can call a completion handler after the user presses a button on the UIAlertView.For example,
UIAlertView *alertView = [UIAlertView alertViewWithTitle:@"Test" buttons:[NSArray arrayWithObjects:@"Ok", @"Cancel", nil] completionHandler:^(int tappedButtonIndex)
{& // do somet